Robert Half
Data Engineer III
SAN RAMON, USA · Mid · Contract
Sponsorship not specified$104k-$153kDetected 15 days ago
PythonCode ReviewSQLDatabricksAWSDockerKubernetesTerraformCI/CDDevOpsPlatform EngineeringMachine LearningSparkData EngineeringData ScienceLLMsAgentic AIMLOpsAI OrchestrationSalesforceTest AutomationUnityLeadershipCollaboration
About the role
- Robert Half provides paid time off for vacation, personal needs, and sick time.
- The amount of Choice Time Off (CTO) our people receive varies based on their years of service and is pro-rated based on the hours worked per week.
Responsibilities
- Solution Design & Technical Leadership
- Lead architecture and design of complex data pipelines on Databricks lakehouse architecture (Unity Catalog, Delta Lake, Structured Streaming)
- Design and build data foundations that enable AI/ML capabilities - feature stores, embedding pipelines, vector search indexes, and model training datasets
- Align data engineering solutions with business strategy, including support for Agentic AI workloads
- Optimize pipeline performance (Delta Lake table layouts, clustering, Z-ordering) and establish monitoring/alerting best practices with clear SLAs
- Build data infrastructure supporting Agentic AI systems - real-time data access layers, context retrieval pipelines, and agent-accessible data services
- Collaborate cross-functionally with DevOps, Platform Engineering, and MLOps roles to integrate data solutions into the broader technology environment and shared AI infratstructure - Mlflow registries, feature stores, and agent orchestration layers
- Provide consultation to Senior Leadership on complex projects and drive continuous improvement initiatives
- Implement data quality strategies (master data management, validation rules, Delta Live Tables expectations) to ensure trust in enterprise data
- 5+ years designing, building, and troubleshooting scalable ETL/ELT pipelines for business-critical production systems
Requirements
- 5+ years with Python and SQL in data engineering for big data ML/analytics workloads
- 3+ years with cloud data services (AWS), container orchestration (Docker, Kubernetes), and IaC (Terraform, CloudFormation)
- 3+ years architecting ML workflows and data platforms with CI/CD, automated testing, and distributed processing (Spark)
- 3+ years collaborating cross-functionally with Data Science, MLOps, Platform Engineering, and DevOps teams
- 3+ years implementing data quality testing and optimizing SQL/Python for cost/performance in the cloud
- 2+ years hands-on with Databricks (Delta Lake, Unity Catalog, Databricks SQL)
- Experience with MLflow experiment tracking and model registry workflows
- Experience designing pipelines that serve AI/ML inference - real-time feature engineering, embedding generation, and context retrieval for LLM-based systems
- Familiarity with Databricks Mosaic AI, Vector Search, and/or Feature Store
Nice to have
- Familiarity with Salesforce/Heroku data infrastructures
- Experience with data virtualization (e.g., Dremio)
- Understanding of Platform Engineering concepts and internal developer platforms
- Experience migrating from legacy data warehouse/lake to unified lakehouse architecture
- Familiarity with Odaseva data security and management
- The specific accommodation requested to complete the employment application.
- $104,000.00 - $153,000.00
- A new hire earns up to 17 days of CTO per calendar year.
Compensation
- The typical annual salary range for this position is shown below and is negotiable depending upon experience and location.
Benefits
- Own health, scalability, and modernization of data infrastructure with Databricks as the strategic platform - including workload migration, compute optimization, and Unity Catalog adoption
Company info
- Robert Half, one of FORTUNE's World's Most Admired Companies and a Fortune 100 Best Companies to Work For is hiring for a Data Engineer III to join the ATI Data Science Innovation department.
Apply directly at Robert Half →Create a free account for alerts like thisView Robert Half immigration profile
This listing is sourced directly from Robert Half's careers page and normalized into a canonical job model.