Artificial Analysis, Inc.
Forward Deployed Engineer - Language Models
San Francisco (On-site) · Mid
Stay score
odds of building a lasting career here
Thin sponsorship signal and lottery-bound. A low-probability bet with your clock running. Prioritize cap-exempt roles and proven entry-level sponsors first.
Lottery odds assume a STEM candidate.
Personalize to your clock →Employer immigration record
from this employer's Department of Labor filings
Files H-1B transfers
Sourced from Department of Labor LCA, PERM and prevailing-wage disclosure data. Employer matching is by name, so figures may be split across an employer's legal entities. Absence of a filing means none appears in our copy of the data, not that none exists.
Community outcomes
No reports yet — be the first to help the next applicant.
About the role
- You'll onboard new models to our evaluation pipeline, run and debug benchmarks, and serve as the primary technical point of contact for AI lab customers: explaining results, fielding methodology questions, and resolving endpoint issues in real time.
- This is a deployed role in the truest sense: you sit at the interface between our platform and our most sophisticated customers, accountable for both sides working.
- It is about running a sophisticated stack exceptionally well, consistently and reliably, while being the trusted engineer our customers ask for by name.
Responsibilities
- Operate and maintain our Python-based language model benchmarking pipeline end-to-end: onboard new models, configure evaluations, execute benchmark runs, and validate results
- Maintain documentation of processes, known issues, and model-specific configurations
- Collaborate with the engineering team to flag pipeline improvements and contribute to process refinements
Requirements
- Strong Python proficiency and comfort working with complex codebases you didn't write
- Hands-on experience working with AI/ML model APIs (OpenAI, Anthropic, Google, Meta, etc.)
- you can trace issues across APIs, data pipelines, and code
- Strong written and verbal English communication skills, with the ability to explain technical concepts clearly to technical stakeholders
Nice to have
- Experience with AI evaluation, benchmarking, or testing methodologies
- Familiarity with LLM inference infrastructure (tokenization, latency measurement, throughput metrics)
- Experience working in or with AI labs or model providers
- Background in B2B SaaS or developer tools
- Why Artificial Analysis?
- Shape how AI gets built: The leading AI labs track our benchmarks and use them to guide their development priorities.
- Your work will directly influence the direction of AI.
- Become a world expert in AI: You will evaluate every major model, across every major capability, as they are released.
Compensation
- Competitive compensation including equity
Company info
- you sit at the interface between our platform and our most sophisticated customers, accountable for both sides working.
- Serve as the primary technical contact for AI lab customers: communicate benchmarking results clearly, explain methodology, field technical questions, and troubleshoot integration issues via Slack and video conferencing
- What We're Looking For
This listing is sourced directly from Artificial Analysis, Inc.'s careers page and normalized into a canonical job model.