Intellivo

Intellivo

Senior Data Engineer

Memphis, Tennessee · Senior

Sponsorship not specifiedDetected 11 days ago
PythonGitSQLAzureCI/CDMachine LearningdbtData EngineeringData ScienceComplianceHIPAACollaborationProblem SolvingOrganizational SkillsProperty & Casualty

About the role

  • This person will work closely with Data Science, ML Engineering, and Software Engineering teams to ensure reliable, governed, and performant data delivery across the organization.
  • Establish alerting and escalation processes for pipeline failures and data anomalies.

Responsibilities

  • The Senior Data Engineer is responsible for designing, building, and optimizing scalable data pipelines and platform infrastructure within a medallion architecture (Bronze, Silver, Gold) on Microsoft Fabric and OneLake.
  • Optimize ingestion workflows for reliability, throughput, and compliance across regulated production environments.
  • Implement error handling, retry logic, and dead-letter patterns to ensure pipeline resilience.
  • Develop Silver layer transformation logic including normalization, deduplication, entity resolution, and schema enforcement within Microsoft Fabric and OneLake.
  • Build Gold layer aggregations and enriched datasets that support ML scoring models and embedded analytics reporting.
  • Implement multi-tenant data isolation patterns including partitioning, access controls, and governed data handling across a large number of client contracts.
  • Document data lineage, transformations, and data contracts to support governance, audit readiness, and operational clarity.
  • Build and maintain data quality validation scripts to detect schema drift, completeness gaps, and business-rule violations across pipeline stages.
  • Partner with ML Engineering and Data Science to deliver features that support model retraining, scoring pipelines, and identification engine capabilities.
  • Collaborate with Software Engineering, Analytics, and business stakeholders to translate operational needs into reliable, production-ready data solutions.

Requirements

  • 7+ years of professional data engineering experience, preferably within Azure-based or Microsoft Fabric environments.
  • Strong programming skills in Python, with advanced SQL experience and data quality validation logic.
  • Experience with Microsoft Fabric, OneLake, Azure Data Factory, or equivalent cloud data orchestration tools.
  • Working knowledge of CI/CD practices for data pipelines and infrastructure-as-code concepts.
  • Demonstrated experience using AI-assisted development tools (e.g., GitHub Copilot, Cursor, or similar) to accelerate pipeline development, code generation, and debugging workflows.
  • Experience with multi-tenant data architectures and governed data handling in regulated environments.
  • Experience with dbt, PySpark, or similar transformation frameworks.
  • Self-directed with a track record of driving projects through completion without constant oversight.
  • Required
  • B.S. or B.A. in Computer Science, Information Systems, Mathematics, or a related field.
  • Hands-on experience designing enterprise data pipelines, ETL/ELT workflows, and medallion or lakehouse architecture patterns.
  • Preferred
  • Familiarity with healthcare data formats (claims, eligibility, EDI 837/835) and HIPAA compliance requirements.
  • Exposure to ML feature engineering, Feature Store design, or data pipelines supporting model training workflows.
  • Professional Skills

Benefits

  • Design and build data ingestion pipelines from multiple structured and unstructured sources including healthcare claims, P&C insurance data, and legal filings into the Bronze layer of the medallion architecture.
  • Maintain Feature Store pipelines that produce machine learning-ready feature sets for model training and inference.
  • Implement monitoring on pipeline health, data freshness, and operational exceptions to maintain high-confidence production data.
Apply directly at Intellivo →Create a free account for alerts like thisView Intellivo immigration profile

This listing is sourced directly from Intellivo's careers page and normalized into a canonical job model.