Speak
Machine Learning Engineer, Assessments
San Francisco
Sponsorship not specifiedDetected 105 days ago
PythonAlgorithmsMachine LearningPyTorchStatisticsResearchLeadershipCollaboration
About the role
- Stand up or improve summative assessments (spoken language ability) and bring them reliably to production
- Help define data needs for training/evaluation (including psychometric measurement needs)
Responsibilities
- Ship and own assessment ML systems end-to-end
- Build, deploy, and maintain scoring models/pipelines (feature extraction → model training → inference → feedback generation)
- Own monitoring, regression tests, and ongoing iteration to maintain accuracy targets
- Implement validation/evaluation frameworks for assessments, including metrics, test sets, and offline/online analysis
- Partner deeply with the Assessment Design Lead
- Co-develop the strategy, together with the Content team, to grow assessments into a core platform at Speak
- Work in a tight weekly loop to deliver incremental improvement
- Drive near-term delivery across products
- Prototype and validate formative assessment approaches to measure improvement over weeks/months
- Support data and labeling strategy
Requirements
- Domain expertise in spoken language proficiency assessment (linguistics, applied linguistics, pedagogy, or equivalent experience)
- Strong experience designing and running evaluation + validation for assessment/scoring systems, and tailoring approaches to a specific product use case
- 4+ years building automatic proficiency assessment systems (or equivalent depth in closely related scoring/evaluation domains)
- PhD is helpful but not required
- Proven ability to ship ML models to production (not only research), including reliability, monitoring, and iteration
Nice to have
- Experience with speech/audio ML
- Experience with psychometrics concepts (reliability/validity, calibration)
- Success depends on a tight loop where constructs/rubrics and model outputs co-evolve - not a sequential handoff.
- You'd join at a magical time when a single person could significantly change the course of the company.
- Do your life's work with people you'll love working with: we care strongly about our craft and want every person at Speak to feel like they're growing every day.
- We believe in the idea that working with people you both enjoy and have respect for makes everything better.
- We hire thoughtfully and only work with people we admire deeply.
- Global in nature: We're live in over 40 countries and launching in a number of new markets soon.
Company info
- Our mission is to reinvent the way people learn, starting with language.
- Learning a language can change a life by opening doors to new cultures, careers, and communities. Two billion people around the world are actively trying to learn a language, but the best way to learn (one-on-one tutoring) is hard to access at scale and hasn't been meaningfully improved in decades. Speak is building a human-level, AI-powered tutor in your pocket: a conversation-first experience that lets learners actually speak, get instant feedback, and progress through carefully designed lessons. The result is a complete path from beginner to confident speaker across multiple languages.
- Speak first launched in South Korea in 2019, where Speak has now become the number one language learning app, and we now serve learners across many markets and 15+ languages. Speak is one of the world's leading AI companies, with over $150m raised in venture investment from OpenAI, Accel, Founders Fund, Khosla Ventures, and more, with a distributed team across San Francisco, Seoul, Tokyo, Taipei, and Ljubljana.
- Learning a language can change a life by opening doors to new cultures, careers, and communities.
- Two billion people around the world are actively trying to learn a language, but the best way to learn (one-on-one tutoring) is hard to access at scale and hasn't been meaningfully improved in decades.
- Speak is building a human-level, AI-powered tutor in your pocket: a conversation-first experience that lets learners actually speak, get instant feedback, and progress through carefully designed lessons.
- The result is a complete path from beginner to confident speaker across multiple languages.
- Speak first launched in South Korea in 2019, where Speak has now become the number one language learning app, and we now serve learners across many markets and 15+ languages.
- Speak is one of the world's leading AI companies, with over $150m raised in venture investment from OpenAI, Accel, Founders Fund, Khosla Ventures, and more, with a distributed team across San Francisco, Seoul, Tokyo, Taipei, and Ljubljana.
- we're growing very quickly, we've most recently raised our Series C from some of the top investors in the valley, and we've achieved product-market fit in our initial markets.
This listing is sourced directly from Speak's careers page and normalized into a canonical job model.