Medical Guardian

Medical Guardian

Principal Data Engineer

Philadelphia, Pennsylvania, United States · Principal

Sponsorship not specifiedDetected 17 days ago
PythonDistributed SystemsCode ReviewSQLDatabricksAzureCI/CDDevOpsPlatform EngineeringMachine LearningSparkData EngineeringRAGIncident ResponseMedical DevicesLeadershipMentoring

About the role

  • Trusted by families, healthcare providers, and care managers, our work is powered by a culture of innovation, compassion, and purpose.

Responsibilities

  • Hands-On Data Engineering and Platform Development Design, build, optimize, and operate production-grade batch and streaming data pipelines on Azure and Databricks, with a primary focus on real-time IoT and telemetry use cases within a Medallion architecture.
  • Develop ETL/ELT workflows to ingest, transform, validate, and serve large volumes of structured, semi-structured, unstructured, and streaming data.
  • Build and maintain reliable data products, data services, APIs, and microservices that support operational applications, analytics, software engineering, and ML/AI teams.
  • Use Python, PySpark, Spark SQL, SQL, Delta Lake, Databricks Workflows, CI/CD, and related tools to build maintainable, testable, and observable data systems.
  • Implement streaming solutions using Azure Event Hubs, Azure Stream Analytics, Databricks, Delta Lake, and related Azure integration patterns.
  • Design cost-effective throughput, partitioning, delivery, retention, and replay strategies for high-volume event and telemetry workloads.
  • Create consumption patterns that support APIs, microservices, operational applications, near-real-time decisioning, analytics, and ML/AI use cases.
  • Databricks, Lakehouse, and Data Platform Architecture Set direction for Databricks-based data engineering patterns, including Medallion architecture, Delta Lake, Spark optimization, data modeling, data quality, and reusable pipeline design.
  • Optimize production Databricks pipelines using PySpark, Spark SQL, Delta Lake, partitioning strategies, caching, shuffle optimization, cluster/job configuration, and cost-aware design.
  • Partner with data platform, security, infrastructure, and engineering teams to ensure the data platform is scalable, secure, reliable, and aligned with enterprise architecture.

Requirements

  • 10+ years of professional experience in data engineering, software engineering, data platform engineering, distributed systems, analytics engineering, or related technical fields.
  • 5+ years of hands-on experience with modern cloud data platforms, including Databricks, Spark, Delta Lake, SQL, Python/PySpark, and production pipeline orchestration.
  • 3+ years of experience leading, managing, mentoring, or providing technical direction to data engineers or related technical teams.
  • Strong experience with Azure cloud services for data engineering, streaming, integration, storage, security, and production operations.
  • Experience designing and operating real-time streaming, event-driven, or near-real-time data pipelines in production or business-critical environments.
  • Experience applying DevOps, CI/CD, testing, version control, code review, documentation, and automation practices to data engineering workloads.
  • Strong understanding of data quality, observability, monitoring, lineage, reliability, cost optimization, privacy, and production support for data systems.
  • Ability to explain data architecture, pipeline behavior, tradeoffs, assumptions, risks, and limitations to both technical and non-technical stakeholders.
  • who will use the data, what decision or workflow does it support, what latency and quality are required, what happens if the data is wrong or late, and how will we know the capability is creating value?

Nice to have

  • 12+ years of relevant professional experience in data engineering, software engineering, data platforms, distributed systems, analytics engineering, commercial software, or production data products.
  • Experience working in commercial software, SaaS, digital products, healthtech, fintech, IoT, consumer technology, or other product-driven environments.
  • Experience with Azure Event Hubs, Azure Stream Analytics, Azure Service Bus, Azure Data Factory, Azure Functions, ADLS, Azure Cosmos
  • This is a hands-on engineering leadership role first.
  • They should also be able to operate with the maturity of a principal-level leader:
  • shaping unclear requirements, making pragmatic technical decisions, managing and mentoring engineers, and driving work forward without waiting for perfect specifications.
  • This is a fast-moving, startup-like environment.
  • We need someone who can move from ambiguous business need to reliable data capability with urgency, discipline, and ownership.

Benefits

  • Medical Guardian is a fast-growing digital health and safety company on a mission to help people live a life without limits.

Company info

  • 5000 list of Fastest Growing Companies, we are redefining what it means to age confidently and independently.
Apply directly at Medical Guardian →Create a free account for alerts like thisView Medical Guardian immigration profile

This listing is sourced directly from Medical Guardian's careers page and normalized into a canonical job model.