Duettoresearch
Staff Software Engineer (Python)
United States · Staff+
Sponsorship not specifiedDetected 54 days ago
PythonJavaGitSQLDatabricksAWSDockerTerraformCI/CDGitHub ActionsDatadogSparkAirflowdbtData EngineeringLLMsEHR/EMRMentoring
About the role
- You'll be the technical authority on Duetto's data lakehouse, driving everything from pipeline architecture and data quality to the shift from batch to near-real-time streaming.
- Trusted by clients ranging from independent boutique hotels to global chains, we've been named the #1 Revenue Management Software by HotelTechAwards four years running and the #1 Best Place to Work in Hotel Tech in 2025.
- Every engineer uses Claude Code and a custom multi-agent system daily.
Responsibilities
- You'll own the design, performance, and reliability of Duetto's data lakehouse - evolving the Python/PySpark pipeline framework across a bronze → silver → gold architecture on AWS, including Glue jobs, Iceberg MERGE operations, schema evolution, and partitioning strategies.
- You'll architect the shift from batch to near-real-time streaming, building SQS-driven stream pipelines with Iceberg sinks and expanding ingestion, normalisation, and analytics layers across the full lakehouse.
- You'll drive data quality and governance at scale - extending the Great Expectations framework, leading adoption of data contracts to formalise schemas between producers and consumers, and owning the Athena SQL layer that analysts and product teams depend on.
- You'll build and maintain shared internal Python libraries published to JFrog, and drive improvements to GitHub Actions, Docker-based testing, and CI/CD deployment workflows.
Requirements
- You may be a good fit if you have:
- Deep expertise in PySpark and distributed data processing - Glue, EMR, or Databricks
- Strong experience with lakehouse architectures: Iceberg, Delta Lake, or Hudi on S3
- Production experience with Airflow or a comparable workflow orchestrator
- Solid AWS production experience across S3, Glue, Athena, Lambda, and SQS
- A track record of improving data quality, governance, and pipeline reliability at scale
- Working knowledge of Java for reading upstream systems
- Experience with Trino or Presto for interactive SQL analytics at scale
- Experience with dbt for data transformation and modelling
- Familiarity with Great Expectations or similar data quality frameworks
Equal opportunity
- Duetto is an equal opportunity employer.
- We celebrate diversity and are committed to creating an inclusive environment for all employees.
- All qualified applicants will receive consideration for employment without regard to race, colour, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, or any other characteristic protected by applicable law.
- Sound like you?
Apply directly at Duettoresearch →Create a free account for alerts like thisView Duettoresearch immigration profile
This listing is sourced directly from Duettoresearch's careers page and normalized into a canonical job model.