Judgmentlabs

Research Engineer

San Francisco

Sponsorship not specifiedDetected 191 days ago

Machine LearningData EngineeringA/B TestingResearch

About the role

Your research will not live on a whiteboard.
You'll work directly with real-world agent data, apply frontier methods in production, and see your work ship immediately into the product.

Build systems to aggregate, index, and analyze large-scale agent interaction data to extract meaningful evaluation signals
Develop agent-based systems for analyzing and evaluating complex, long-running behaviors
Design and implement post-training and optimization workflows to improve agent behavior
Build internal tools and infrastructure to support rapid experimentation, analysis, and training
Agents can't work without this. Today's agents hallucinate, drift, and break in production. We're building the infrastructure that fixes this: the monitoring layer that makes agents self-improving.
We're wired to win. We're a team of less than 20 but we ship like 50+ on the daily. You'll be working with olympiad medalists, debate champions, and competitive athletes who bring that same intensity to company building.
Judgment Labs builds infrastructure for Agent Behavior Monitoring (ABM).
Hundreds of teams building autonomous agents rely on Judgment to understand how their systems are behaving post-deployment.

You are comfortable working across infrastructure and systems, spanning training, data pipelines, and model serving.
You are comfortable working across teams to translate research into product, balancing real-world customer constraints and tradeoffs.

Our investors include Lightspeed, SV Angel, Valor Equity Partners, Nova Global, Chris Manning, Michael Ovitz, Michael Abbott, Cory Levy, Kevin Hartz, and others.
You have a strong background in reinforcement learning, agents, or machine learning fundamentals

This listing is sourced directly from Judgmentlabs's careers page and normalized into a canonical job model.