Artificial Analysis, Inc.
Member of Technical Staff (Language Model Evaluations)
San Francisco (On-site) · Staff+
Stay score
odds of building a lasting career here
Thin sponsorship signal and lottery-bound. A low-probability bet with your clock running. Prioritize cap-exempt roles and proven entry-level sponsors first.
Lottery odds assume a STEM candidate.
Personalize to your clock →Employer immigration record
from this employer's Department of Labor filings
Files H-1B transfers
Sourced from Department of Labor LCA, PERM and prevailing-wage disclosure data. Employer matching is by name, so figures may be split across an employer's legal entities. Absence of a filing means none appears in our copy of the data, not that none exists.
Community outcomes
No reports yet — be the first to help the next applicant.
About the role
- Language model evaluation is the sharpest question in AI: what can these systems actually do?
- Our answers, from the Artificial Analysis Intelligence Index to AA-Omniscience, AA-Briefcase and our coding agent evaluations, are the reference the industry uses.
- You will run your work across every major model as it releases and publish results the whole industry reads.
Responsibilities
- Design Next-Generation Frontier Evals: Conceive and ship the next generation of frontier evaluations, like AA-Briefcase and AA-Omniscience, across reasoning, knowledge, coding, agentic capability and beyond
- Build Evaluation Datasets and Infrastructure: Construct the datasets, harnesses and scoring systems behind our benchmarks, engineered for contamination resistance and repeatability at frontier scale
- Shape the Future Intelligence Index: The evaluations you build will contribute to future versions of the Artificial Analysis Intelligence Index and other areas of our platform, defining how the industry measures frontier capability
- our commercial team owns client relationships day to day, so your time stays on the science
- Evaluate Every Major Model: Run our evaluation suite across frontier releases as they land, and own the integrity of the results the industry quotes
- Become AI-Native: Embrace an AI-native workflow, using cutting-edge AI tools to generate leverage in a fast-changing industry and maintain our competitive edge in AI benchmarking
Requirements
- You have deep, hands-on experience evaluating language models and strong opinions about why most benchmarks fail.
- 3+ years of relevant professional experience, across industry or research
- Strong statistical grounding: you know when a result is signal and when it is noise
- Genuine, demonstrable interest and knowledge of Frontier AI.
Compensation
- Competitive compensation including equity
Company info
- We are the go-to authority for understanding AI, from AI labs and enterprises to media, investors, and policymakers.
- What We're Looking For
This listing is sourced directly from Artificial Analysis, Inc.'s careers page and normalized into a canonical job model.