Bounteous
Senior Databricks Engineer
Canada · Senior
Work authorization requiredCAD 91k-CAD 119kDetected 29 days ago
PythonCode ReviewGitSQLDatabricksAWSGCPAzureCloud PlatformsCI/CDMachine LearningSparkAirflowdbtData EngineeringData ScienceNLPLLMsRAGComplianceAccessibilityUnityHIPAA
About the role
- Our clients worldwide benefit from the skills and expertise of over 4,000+ expert team members across the Americas, APAC, and EMEA.
Responsibilities
- Architect, build, and maintain scalable ETL/ELT pipelines on the Databricks Lakehouse Platform using PySpark, Spark SQL, and Delta Lake.
- Design and implement medallion (bronze/silver/gold) data architectures and enforce data quality, governance, and lineage standards.
- Optimize Spark jobs and cluster configurations for performance and cost, including partitioning, caching, and autoscaling strategies.
- Implement and manage Unity Catalog for access control, data governance, and cross-workspace asset sharing.
- Build and orchestrate workflows using Databricks Workflows, Delta Live Tables, and CI/CD pipelines.
- Monitor production pipelines, troubleshoot failures, and drive root-cause analysis and continuous improvement.
- Collaborate with data scientists, analysts, and business stakeholders to translate requirements into reliable data products.
Requirements
- Expert proficiency in Python ( PySpark ) and advanced SQL.
- Deep hands-on experience with Delta Lake, Unity Catalog, and the medallion architecture pattern.
- Strong experience with at least one major cloud platform (AWS, Azure, or GCP) and its core data services.
- Proven track record optimizing Spark performance and managing cluster cost.
- Experience with data modeling, warehousing concepts, and building dimensional/analytics-ready datasets.
- Proficiency with Git-based version control, CI/CD, and infrastructure-as-code.
- Bachelor's degree in Computer Science, Engineering, or equivalent practical experience.
Nice to have
- Databricks certification (Data Engineer Associate/Professional).
- Experience with Delta Live Tables, structured streaming, and real-time data processing.
- Experience with orchestration tools (Airflow, dbt ) and data observability platforms.
- Exposure to data governance, security, and compliance frameworks (e.g., GDPR, HIPAA, SOC 2).
- Hands-on experience using AI coding assistants (e.g., Claude Code, GitHub Copilot, Cursor) to accelerate development, refactoring, and code review.
- Familiarity with large language model APIs and SDKs (e.g., Anthropic Claude, OpenAI) and prompt engineering for data and analytics use cases.
- Experience integrating GenAI capabilities into data pipelines or applications, including retrieval-augmented generation (RAG) and vector search.
- Awareness of responsible AI practices, including evaluation, guardrails, and cost/latency trade-offs when deploying LLM-based solutions.
Skills
- Ensure data protection measures are integrated throughout the information lifecycle to safeguard sensitive information
Compensation
- CAD 91k-CAD 119k
Visa & Work Authorization
- Bounteous is willing to sponsor eligible candidates for employment visas
Apply directly at Bounteous →Create a free account for alerts like thisView Bounteous immigration profile
This listing is sourced directly from Bounteous's careers page and normalized into a canonical job model.