Together AI

Staff Software Engineer, GPU Infrastructure Lifecycle Management

San Francisco · Staff+ · Full-time

Sponsorship not specified$240k-$280kDetected 5 days ago

PythonGoRustAlgorithmsCode ReviewKubernetesAnsibleCI/CDTemporalKafkaMachine LearningAI OrchestrationCadenceEmbedded SystemsBGP/OSPFResearch

About the role

The Research and Inference team is your customer: today they file tickets and wait
the target state is that they issue a single API call to stand up, scale, or tear down a cluster, and the system takes care of the rest.
The platform is manifest-driven such that teams declare the desired state of a cluster or host - shape, topology, software stack - and the system is responsible for reconciling reality to that manifest, continuously, through every stage of its lifecycle.

Build the self-service API: design declarative APIs and a control plane so the inference team can request, scale, and tear down inference clusters with one API call - no ticket, no human in the loop.
Own reliability of the pipeline: idempotency, retries, rollback, and drift detection so the provisioning system is as dependable as any other production service.
Partner with the inference/ML platform team: understand the cluster shapes they need - topology, interconnect, scheduling constraints - and encode them as first-class abstractions in the platform.
You will design the engines that manifest the schema, the engines that execute against it, and the workflows that carry a piece of hardware or a cluster from one state to the next-taking it from bare metal to a fully functioning AI cluster for training or inference.
You build it, you own it.

Experience with durable workflow orchestration tools such as Temporal, Cadence, or equivalent to run long-lived, manifest-driven workflows that survive failures and resume mid-execution.
Experience with event-driven systems - designing and building software around message queues, event streams, or pub/sub (e.g., Kafka, NATS, SQS) rather than polling or cron-driven scripts.
You've built internal platforms or APIs consumed by other engineering teams and care about the developer experience of what you ship.
You are not only responsible for delivering the software but also for operating and supporting it in production.

Experience with GPU cluster software stacks (NCCL, CUDA, InfiniBand/RoCE).
Prior work at a hyperscaler, GPU cloud, or datacenter-scale infrastructure organization.
Systems programming in Rust or Go.
About Together AI
Together AI is a research-driven artificial intelligence company.
We have contributed to leading open-source research, models, and datasets to advance the frontier of AI, and our team has been behind technological advancement such as FlashAttention, Hyena, FlexGen, and RedPajama.
Individual compensation will be determined by experience, skills, and job-related knowledge.
Equal Opportunity

We offer competitive compensation, startup equity, health insurance and other competitive benefits.
The US base salary range for this full-time position is: $240,000 - $280,000 + equity + benefits.
Our salary ranges are determined by location, level and role.
Individual compensation will be determined by experience, skills, and job-related knowledge.

Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.

t opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more

This listing is sourced directly from Together AI's careers page and normalized into a canonical job model.