Sonatus
Senior Manager, Engineering - AI Validation
Sunnyvale, CA · Senior
Sponsorship not specified$240k-$290kDetected 30 days ago
PythonBashVector DatabasesCloud PlatformsJenkinsLinuxMachine LearningData EngineeringLLMsRAGAgentic AILLMOpsLangGraphSystems EngineeringElectrical EngineeringPlaywrightpytestManual Testing
Stay score
odds of building a lasting career here
59Sponsors, lottery-bound
Cap-exempt (no lottery)0
Sponsors this role80
Entry-level history0
PERM / green-card track0
Lottery odds (Level IV)94
Fits your clock70
Sponsors, but it's cap-subject — you still face the weighted lottery (~61% per draw at Level IV). Good if you win; have a cap-exempt backup on your list.
Lottery odds assume a STEM candidate.
Personalize to your clock →Employer immigration record
from this employer's Department of Labor filings
Green-card filing pattern in this occupation
Context, not a finding about this posting: of this employer's 5 green-card filings in this occupation, 100% were for a worker who already held the job.
Files H-1B transfers
7 transfer filings in the last year, covering 7 workers. Median labor-condition decision: 7 days. An employer that already files transfers is one that can take over an existing H-1B.
Sourced from Department of Labor LCA, PERM and prevailing-wage disclosure data. Employer matching is by name, so figures may be split across an employer's legal entities. Absence of a filing means none appears in our copy of the data, not that none exists.
Community outcomes
No reports yet — be the first to help the next applicant.
About the role
- Establish AI governance, Responsible AI practices, model lineage, safety guardrails, and compliance processes appropriate for automotive safety-critical systems and enterprise AI products.
- Act as the quality gatekeeper for AI-enabled releases, partnering with engineering, product, safety, and OEM stakeholders to identify risks, define release criteria, and ensure production readiness.
Responsibilities
- Define and drive Sonatus's AI validation strategy across embedded, in-vehicle, cloud-connected, and cloud-native AI systems, identifying gaps in model development, testing, deployment, and governance.
- Lead, hire, mentor, and grow a high-performing AI Validation organization, establishing scalable engineering processes, technical direction, and execution excellence.
- Own the end-to-end validation strategy for AI/ML models, LLMs, RAG pipelines, and agentic AI workflows-from data pipelines and model training through cloud services and in-vehicle deployment.
- Design and maintain evaluation harnesses for RAG and agentic systems that measure retrieval quality, grounding, citation accuracy, factual consistency, context relevance, safety, latency, reliability, and execution correctness.
- Evaluate, integrate, and optimize open-source and commercial AI validation technologies, driving build-versus-buy decisions for Sonatus's AI quality platform.
- Collaborate across engineering teams to define validation strategies for emerging AI capabilities, rapidly prototype new evaluation approaches, and standardize successful practices into reusable frameworks.
- Drive continuous improvement by tracking industry advances in AI evaluation, agentic AI, LLM validation, and RAG systems, translating them into scalable validation capabilities across Sonatus.
Requirements
- 10+ years of experience in software or systems engineering-including embedded, cloud, networking, security, or automotive domains-with 3+ years leading high-performing engineering or QA organizations.
- Proven experience designing and implementing scalable evaluation frameworks for AI systems, including multi-step agentic workflows, regression testing, benchmarking, and automated quality scoring.
- Practical experience with RAG evaluation frameworks such as RAGAS, including evaluation tuning, embedding optimization, retrieval quality improvement, and production-scale LLM evaluation pipelines.
- Experience validating hallucination, grounding, citation accuracy, bias, fairness, toxicity, and factual consistency in production LLM applications.
- Experience designing systems that verify external knowledge claims and ensure responses are grounded in traceable citations and trusted data sources.
- Strong experience testing cloud-native platforms and cloud-managed embedded products, including end-to-end system validation.
- Experience establishing AI governance, safety, compliance, and Responsible AI practices for enterprise or safety-critical systems.
- Proficiency in Python, Linux, shell scripting, modern test frameworks (PyTest, Playwright, Behave), and engineering productivity tools such as Jenkins and JIRA.
- Experience implementing enterprise-grade AI governance (auditability, monitoring, policy enforcement) in production systems.
- Hands-on experience developing, deploying, testing, or operating AI/ML systems, with strong expertise in modern ML workflows, neural networks, and MLOps.
- Deep understanding of LLMs, RAG architectures, vector databases, embeddings, retrieval optimization, and agentic AI frameworks such as LangGraph or equivalent orchestration platforms.
- Strong expertise with hybrid evaluation methodologies combining deterministic validation (citation grounding, structural validation, exact matching) and probabilistic LLM-as-a-Judge techniques (faithfulness, answer relevance, context precision, task completion).
- Ways to Stand Out:
- Experience validating AI or agentic systems in safety-critical or regulated industries (automotive, aerospace, medical).
Nice to have
- Bachelor's degree in Computer Science, Computer Engineering, Electrical Engineering, or a related field required (MS preferred).
Skills
- That's why leading OEMs trust Sonatus to accelerate this shift.
- Our technology is already in production across more than 9 million vehicles on the road today and rapidly expanding.
- The Opportunity:
Compensation
- $240,000 - $290,000 USD
Benefits
- Health care plan (Medical, Dental & Vision)
- Flexible and Dependent Care Expense program
- Retirement plan (401k)
- Life Insurance (Basic, Voluntary & AD&D)
- Unlimited paid time off per year, 14+ paid holidays
- Wellness benefit allowance
- Phone & Internet reimbursement
Company info
- At Sonatus, we're driving the transformation to AI-enabled software-defined vehicles.
Apply directly at Sonatus →Create a free account for alerts like thisView Sonatus immigration profile
This listing is sourced directly from Sonatus's careers page and normalized into a canonical job model.