Arcada
Member of Technical Staff, ML Research Engineer
San Francisco · Staff+
Sponsorship not specifiedDetected 89 days ago
Machine LearningLLMsResearch
About the role
- Your work will shape our public leaderboards and the evaluation tools we share with frontier labs.
- You'll work at the intersection of engineering, ML, and research - deciding what to evaluate, how to evaluate it (using real human preference data and other signals), and how to turn those results into better rankings and insights.
- Turn human preference votes and interaction traces into reliable signals about model capability, taste, reasoning, robustness, and agent behavior
Responsibilities
- We create the evolutionary pressure that pushes models toward what people actually want.
- You'll design and run experiments that turn millions of human preference into reliable signals about what makes models useful, trustworthy, and capable in practice (design taste, agent behavior, multi-step tasks, reasoning, etc.).
- Design and run large-scale evaluations that measure how frontier models perform in real-world workflows
- Develop ranking systems, analysis pipelines, and experimental methods for comparing models
Requirements
- Experience training, fine-tuning, or evaluating models, including LLMs, reward models, preference models, or RLHF/DPO-style systems
- Ability to turn vague real-world problems into concrete evaluation tasks, experiments, and measurable systems
Skills
- Work with engineers to turn research findings into user-facing products, leaderboards, and tools for frontier labs
Company info
- At Arcada Labs, we build products used by millions of people around the world that give us direct access to real human preference and judgment.
- We're looking for an ML Research Engineer to help us build better ways to evaluate and understand real AI capabilities.
This listing is sourced directly from Arcada's careers page and normalized into a canonical job model.