Build Group

AI Engineer - Harness & Evals

San Francisco · Mid

Sponsorship not specifiedDetected 76 days ago

PythonDistributed SystemsVector DatabasesLLMsRAGAgentic AIFinancial ModelingSystems EngineeringResearchCollaboration

About the role

This is a hands-on engineering role for someone who wants to make agents reliable, observable, scalable, and safe enough for high-stakes real-world workflows.
It is infrastructure work for production AI systems.

Build the core agent platform used by product engineers to create, run, evaluate, debug, and deploy AI workflows.
Design infrastructure for long-running agents, tool orchestration, workflow state, retries, fallbacks, human handoff, and resumability.
Build context and retrieval systems that help agents use the right documents, structured data, prior decisions, project state, and tool outputs.
Create eval infrastructure for agent behavior, document understanding, groundedness, workflow completion, visual reasoning, latency, cost, and regressions.
Build observability systems for traces, prompts, model versions, tool calls, intermediate reasoning artifacts, failure modes, human overrides, and production quality metrics.
Partner with product engineers to turn repeated workflow patterns into reusable primitives, SDKs, templates, and platform capabilities.
Own performance, scalability, security, and maintainability across the AI platform.
Help define the engineering standards for production agent systems at Build.
Build an agent runtime that supports durable execution, resumable workflows, retries, tool permissions, human approval gates, and production traceability.
Build an eval platform where engineers can run offline tests, replay production traces, compare model and prompt changes, detect regressions, and review failure clusters.

You have deep experience with backend systems, distributed systems, data systems, workflow engines, observability, or developer platforms.
Experience with agentic frameworks, LLMs, workflow engines, vector databases, reranking, model gateways, or AI observability tools.
Experience with document AI, multimodal systems, structured extraction, citation systems, or knowledge graph infrastructure.

Meaningful equity Ownership in the work is matched with ownership in the company.

alternative asset investors, developers, infrastructure owners, energy companies, industrial operators, and public-sector partners.
Their work shapes the physical world, but the workflows behind that work are still slow, fragmented, document-heavy, and dependent on expert coordination.
We believe the next generation of built-world software will not just organize work.
It will help do the work.
Agents will reason across documents, drawings, financial models, market data, approvals, constraints, and expert judgment.
Human experts will stay in control, but they will operate with far more leverage.
We are backed by leading investors and operators, including executives from Blackstone and OpenAI, alongside top venture firms.
We are building a generational company at the intersection of AI and the physical world.

This listing is sourced directly from Build Group's careers page and normalized into a canonical job model.