Mercor

Mercor

Research Scientist, APEX Benchmarks

San Francisco

Sponsorship not specifiedDetected 50 days ago
Machine LearningNLPLLMsAccountingCorporate LawResearchLab ResearchCommunication

Stay score

odds of building a lasting career here

40Risky
Cap-exempt (no lottery)0
Sponsors this role90
Entry-level history0
PERM / green-card track0
Lottery odds40
Fits your clock70

Thin sponsorship signal and lottery-bound. A low-probability bet with your clock running. Prioritize cap-exempt roles and proven entry-level sponsors first.

Lottery odds assume a STEM candidate.

Personalize to your clock →

Community outcomes

No reports yet — be the first to help the next applicant.

About the role

  • APEX is the AI Productivity Index: a family of benchmarks that measures whether frontier models can do economically valuable professional work.
  • APEX-1 covers single-turn tasks across investment banking, corporate law, consulting and medicine.

Responsibilities

  • Benchmark design: Decide what the next APEX benchmark should measure, based on frontier model performance, saturation of existing evals, and where economically valuable work is still out of reach.
  • Own the task taxonomy, difficulty calibration, contamination controls and statistical design.
  • Dataset design: Design expert-built datasets and grading rubrics at scale - deciding what makes a task hard, what makes a grade defensible, and how to hold quality while thousands of experts produce work in parallel.
  • Partnerships: Work with academic collaborators and industry partners (as we did with Cognition on APEX-SWE) to co-design benchmarks and get them adopted.
  • Cross-functional work: Partner with data operations, engineering, product and strategy to take a benchmark from design to production, and to surface research findings that shape the company roadmap.
  • Stay at the frontier: Track the LLM evaluation literature and bring what's good into how Mercor builds benchmarks.
  • Research background in evaluation: Strong applied or academic research background in LLM evaluation, benchmarking, NLP or a related field, with a track record of rigorous experimental design.
  • APEX results are read by the labs building the models.

Requirements

  • Statistical rigor: You reason carefully about sampling, variance, contamination and grader reliability, and you don't ship a number you can't defend.
  • Exceptional communication: You can present complex technical findings clearly to frontier lab researchers and to non-technical audiences, in writing and in person.
  • Comfort with ambiguity: You've operated in fast-moving, cross-functional environments where the problem space is not yet defined.
  • In-person: You are excited to work from our San Francisco office five days a week in a high-intensity, high-ownership environment.

Nice to have

  • equivalent industry or frontier lab research experience considered.
  • Publications at top-tier venues (NeurIPS, ICML, ACL, ICLR), especially in evaluation, benchmarking or data-centric AI.
  • Experience authoring a widely-adopted public benchmark or dataset.
  • Industry experience on an evaluation, benchmarking or post-training team at a frontier lab.
  • Domain depth in one of APEX's professional verticals - finance, law, consulting, accounting, medicine or software engineering.
  • Experience designing rubrics, model-as-judge pipelines, or human annotation programs at scale.

Skills

  • Statistical rigor:
  • Strong coding skills.
  • Exceptional communication:
  • Comfort with ambiguity:
  • Interest in the commercial side:
  • APEX-Agents tests multi-hour, cross-application agentic work in real tools.

Compensation

  • Bi-annual performance bonus structure

Benefits

  • Bi-annual performance bonus structure
  • Generous equity grant vested over 4 years
  • Up to $15k Relocation bonus
  • $10K housing bonus (if you live within 0.5 miles of our office)
  • $1.5K monthly stipend for meals
  • $200 monthly laundry reimbursement
  • $200 monthly personal wellness reimbursement
  • Health, Dental, Vision insurance

Company info

  • Mercor's mission is to organize human intelligence to power the AI economy.
  • We're a leading AI data company, building the layer between human expertise and frontier models.
  • Millions of domain experts on the platform are paid over $4 million per day to train frontier AI models.
  • Mercor's APEX benchmark family measures AI's real-world impact on professional work.
  • Mercor Enterprise brings this same infrastructure to Fortune 500 companies:
  • helping companies capture how their best people actually work, translating that expertise directly back into agents.
  • Mercor is creating a new category of work where expertise powers AI advancement.
  • Achieving this requires an ambitious, fast-paced and deeply committed team.
  • You'll work alongside researchers, operators, and AI companies at the forefront of shaping the systems that are redefining society.
  • Mercor is a profitable Series C company valued at $10 billion.
  • We work in-person five days a week in our San Francisco, NYC, or London offices.
  • This is a highly visible role at the intersection of research, company strategy, and go-to-market.
  • External research voice: Publish - arXiv papers, open datasets, blog posts, conference talks, leaderboard releases - and represent Mercor's research in conversations with frontier labs, customers and the press.
Apply directly at Mercor →Create a free account for alerts like thisView Mercor immigration profile

This listing is sourced directly from Mercor's careers page and normalized into a canonical job model.