Harvey
Senior Engineering Manager, Model Infrastructure
San Francisco · Senior
Sponsorship not specified$272k-$355kDetected 14 hours ago
Distributed SystemsAzureCloud PlatformsKubernetesKafkaMachine LearningSparkAirflowData EngineeringLLMsAgentic AIA/B TestingIncident ResponseResearchLeadershipCommunication
About the role
- Model Infrastructure is one of Harvey's most strategic engineering organizations.
Responsibilities
- Lead and grow a high-performing team of software engineers responsible for Harvey's Model Infrastructure platform.
- Build highly reliable systems for model provisioning, capacity management, failover, and incident response across multiple AI providers.
- Own Harvey's multi-provider model platform, including provider integrations, SDK upgrades, API migrations, and onboarding new model providers.
- Partner with Product Engineering to support new model launches, capacity planning, experimentation, and proactive production monitoring.
- Lead initiatives to improve inference efficiency, reduce infrastructure costs, and increase model utilization across providers.
- Build the infrastructure foundation for Harvey's future model training efforts, including data pipelines, model operations, training environments, and AI platform capabilities.
- Partner with executive leadership on long-term AI infrastructure strategy and vendor relationships.
- Recruit, mentor, and develop exceptional engineering talent while fostering a culture of technical excellence and operational ownership.
Nice to have
- Experience working with multiple model providers such as OpenAI, Anthropic, Azure OpenAI, Fireworks, Baseten, or open-source model ecosystems.
- Experience with Kubernetes, cloud infrastructure, distributed systems, and large-scale observability platforms.
- Experience supporting GPU infrastructure, model training platforms, or ML infrastructure.
- Familiarity with data platforms and technologies such as Spark, Kafka, Flink, Airflow, or Iceberg.
- Experience leading organizations through periods of rapid growth and technical transformation.
- YOU CAN FIND ALL OF OUR APPLICANT PRIVACY NOTICES HERE https://harveyai.notion.site/Harvey-Candidate-Privacy-Notices-319ac3fcdd7a803bb807d5094f249922?pvs=74.
Compensation
- $272,000 - $355,000 USD
Benefits
- Drive the evolution of our Unified Model Controller (UMC) and Model Selector platform to automatically detect degraded models and intelligently route traffic based on health, latency, quality, compliance, and cost.
- Improve observability through health dashboards, alerting, token usage analytics, cost reporting, and end-to-end model telemetry.
This listing is sourced directly from Harvey's careers page and normalized into a canonical job model.