Magic
Member of Technical Staff, Evals
San Francisco · Staff+
Sponsorship not specified$200k-$550kDetected 50 days ago
Machine LearningResearch
About the role
- Sweating the details matters on this team.
- We care deeply about correctness, reproducibility, and measurement quality.
- Evals are essential to the success of the company.
Responsibilities
- As a Member of Technical Staff on Evals, you will build both the platform and the evaluations themselves.
- You'll develop infrastructure for large-scale evaluations, data ablations, and dataset quality analysis, while designing and validating the methodologies used to measure model performance.
- Evals are essential to the success of the company. By building trustworthy evaluation systems, you will help Magic make better research decisions, build better datasets, and ship better products.
- Build and maintain the internal evals platform used across Magic
- Develop infrastructure for running large-scale evaluations
- Build systems to measure dataset quality and identify opportunities to improve training data
- Partner with research, data, inference, and product teams to define metrics that accurately reflect model quality
- By building trustworthy evaluation systems, you will help Magic make better research decisions, build better datasets, and ship better products.
- Visa sponsorship and relocation support for candidates moving to San Francisco
- Hands-on. At Magic, everyone is building
Requirements
- Track record of owning technical projects end-to-end
- Ability to reason critically about benchmarks, metrics, and experimental methodology
- Experience designing, implementing, or operating systems that run at scale
Compensation
- Annual salary range between $200K - $550K depending on experience
- 401(k) plan with 6% salary matching
Benefits
- Equity is a significant part of total compensation, in addition to salary
- Generous health, dental, and vision insurance for you and your dependents
- Unlimited paid time off
- Design, implement, and validate eval tasks for pre-training, post-training, reinforcement learning, inference, and product systems
Company info
- The team supports pre-training, post-training, data, inference, and product, and sits on the critical path of many of the company's most important decisions.
- Magic's mission is to build safe AGI that accelerates humanity's progress on the world's most important problems.
Visa & Work Authorization
- Visa sponsorship and relocation support for candidates moving to San Francisco
This listing is sourced directly from Magic's careers page and normalized into a canonical job model.