Scribd
Senior Software Engineer (Python + Distributed systems)
San Francisco · Senior
Sponsorship not specifiedDetected 247 days ago
PythonScalaDistributed SystemsBackend DevelopmentDatabricksAWSTerraformDatadogMachine LearningSparkAirflowData EngineeringData ScienceLLMsAI OrchestrationResearchLeadershipCollaborationMentoring
About the role
- This role offers the opportunity to work on cutting-edge generative AI and metadata enrichment problems at a truly global scale.
- Our backend systems are primarily built in Python, leveraging AWS services such as Lambda, ECS, SQS, and ElastiCache for event-driven and distributed processing.
- We also use Airflow, Spark, Databricks, Terraform, and Datadog for orchestration, data processing, and observability.
Responsibilities
- Lead the design, implementation, and scaling of event-driven, distributed systems to extract, enrich, and process metadata from large-scale document and media datasets.
- Partner with Data Science, Infrastructure, ML Engineering, and Product teams to architect and deliver robust systems that balance scalability, high performance, and rapid iteration.
- Build and maintain scalable APIs and backend services for high-throughput content processing.
- Leverage AWS services (ECS, Lambda, SQS, ElastiCache, CloudWatch) to design and deploy resilient, high-performance systems.
- Optimize and refactor existing backend systems for scalability, reliability, and performance.
- In this role, you'll design and optimize large-scale data and service pipelines running on AWS, supporting Scribd's content enrichment and metadata systems.
- We process hundreds of millions of documents, billions of images, and deliver high-quality metadata to enable content discovery and trust for millions of users worldwide.
- We encourage people of all backgrounds to apply, and believe that a diversity of perspectives and experiences create a foundation for the best ideas.
- Come join us in building something meaningful.
Requirements
- 7+ years of professional software engineering experience with a focus on backend or distributed systems development.
- Expertise in designing and architecting large-scale event-driven and distributed systems
- Strong cloud expertise with AWS services (ECS, Lambda, SQS, SNS, CloudWatch, etc.).
- Experience with infrastructure-as-code tools like Terraform.
- Experience leading technical projects and mentoring engineers
- Bachelor's degree in Computer Science or equivalent professional experience.
Nice to have
- Familiarity with data processing frameworks (Spark, Databricks) and workflow orchestration tools.
- Experience integrating ML or LLM-based models into production systems.
- San Francisco is our highest geographic market in the United States.
- We carefully consider a wide range of factors when determining compensation, including but not limited to experience
- job-related skill sets
- and other business and organizational needs.
- Employees must have their primary residence in or near one of the following cities.
- Strong proficiency in Python (5+ years).
Skills
- This posting reflects an approved, open position within the organization.
Compensation
- At Scribd, Inc., your base pay is one part of your total compensation package and is determined within a range.
Benefits
- Comprehensive health, dental, and vision coverage
- Mental health support and disability coverage
- Generous paid time off, including vacation, sick time, holidays, winter break, volunteer time, and sabbaticals
- Paid parental leave and family support benefits
- Retirement matching and employee equity
- Learning and development programs and professional growth opportunities
- Wellness and home office stipends
- Scribd Flex (flexible work model)
Company info
- The ML Data Engineering team powers metadata extraction, enrichment, and content understanding across all Scribd brands.
- Our systems operate at massive scale, supporting diverse datasets like user-generated content (UGC), ebooks, audiobooks, and more.
- We work at the intersection of machine learning, data engineering, and distributed systems, collaborating closely with applied research and product teams to deploy scalable ML and LLM-powered solutions in production.
- At Scribd, Inc., your base pay is one part of your total compensation package and is determined within a range.
This listing is sourced directly from Scribd's careers page and normalized into a canonical job model.