Aircall.io, Inc.

Aircall.io, Inc.

Machine Learning Engineer (Evals and Voice Models)

San Francisco Office · Mid

Sponsorship not specified$181k-$250kDetected 36 days ago
Machine LearningDeep LearningLLMsRAGAgentic AIStatisticsA/B TestingCRMManual TestingCommunicationMentoring

Stay score

odds of building a lasting career here

61Sponsors, lottery-bound
Cap-exempt (no lottery)0
Sponsors this role90
Entry-level history0
PERM / green-card track0
Lottery odds (Level IV)94
Fits your clock70

Sponsors, but it's cap-subject — you still face the weighted lottery (~61% per draw at Level IV). Good if you win; have a cap-exempt backup on your list.

Lottery odds assume a STEM candidate.

Personalize to your clock →

Employer immigration record

from this employer's Department of Labor filings

Files H-1B transfers

16 transfer filings in the last year, covering 16 workers. Median labor-condition decision: 7 days. An employer that already files transfers is one that can take over an existing H-1B.

Sourced from Department of Labor LCA, PERM and prevailing-wage disclosure data. Employer matching is by name, so figures may be split across an employer's legal entities. Absence of a filing means none appears in our copy of the data, not that none exists.

Community outcomes

No reports yet — be the first to help the next applicant.

About the role

  • We're redefining customer communications by bringing voice, SMS, WhatsApp, and AI together into one seamless workspace.
  • Our momentum comes from a simple idea: help teams work smarter, not harder.
  • The result is higher revenue, faster resolutions, and teams that scale with confidence.

Responsibilities

  • Design and document comprehensive evaluation frameworks for Aircall's AI agents across voice, chat and messaging.
  • Analyze system design decisions and identify strengths, weaknesses, and potential failure points.
  • Design annotation guidelines and workflows for human-labeled evaluation data, and calibrate LLM-as-judge systems against human raters to ensure automated evals stay trustworthy over time.
  • Own the metric contract for every published AI metrics, including definition, population, grain, rollup, validity window.
  • Build release gates, the offline regression suite each AI surface must pass before a prompt, model, or config change ships, measuring reliability across repeated trials, not just average pass rates.
  • Build voice-specific evaluation: simulated callers across accents, languages, background noise, barge-in, DTMF, and tool failures, with latency and ASR accuracy as first-class quality metrics.
  • We're working to create a place filled with diverse people who can enrich and learn from one another.
  • We're committed to ensuring that everyone not only has a seat at the table but is valued and respected at it by providing equal opportunities to develop and thrive.
  • We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses.

Requirements

  • 3+ years of experience in ML Engineering or Applied ML with 8+ years of overall experience
  • Hands-on experience in failure analysis and evaluating LLMs

Nice to have

  • Experience evaluating LLMs or agentic systems (e.g., LLM-as-a-judge, RAG evaluation)
  • Experience with synthetic data generation and prompt engineering
  • Experience training or fine-tuning voice models at scale, with familiarity in synthetic data generation, model distillation, or low-latency inference optimization for production voice agents.
  • 🚀 Key moment to join Aircall in terms of growth and opportunities
  • 💆‍♀️ Our people matter, work-life balance is important at Aircall
  • 🌍 45+ Nationalities: cosmopolite & multi-cultural mindset
  • We pride ourselves on promoting active inclusion within our business to foster a strong sense of belonging for all.
  • Above all else, we understand and acknowledge that we have work to do and much to learn.

Compensation

  • $181,000 - $250,000 USD
  • 💶 Competitive salary package & benefits

Benefits

  • 📚 Fast-learning environment, entrepreneurial and strong team spirit
  • At Aircall, we believe diversity, equity and inclusion - irrespective of origins, identity, background and orientations - are core to our journey.
  • We will constantly challenge ourselves to make sure that we live up to our ambitions around diversity, equity and inclusion, and keep this conversation open.

Company info

  • We've built a product customers love and a business that's scaling quickly, backed by world-class investors and driven by rapid AI innovation across multiple product lines.
  • Build and maintain live quality monitoring for deployed AI agents, tracking accuracy, resolution rate, and safety signals in production, and flagging model or data drift before it impacts customers.
Apply directly at Aircall.io, Inc. →Create a free account for alerts like thisView Aircall.io, Inc. immigration profile

This listing is sourced directly from Aircall.io, Inc.'s careers page and normalized into a canonical job model.