Thinking Machines Lab
Software Engineer, Production Inference (Distributed Inference)
San Francisco · Mid
Stay score
odds of building a lasting career here
Sponsors, but it's cap-subject — you still face the weighted lottery (~61% per draw at Level IV). Good if you win; have a cap-exempt backup on your list.
Lottery odds assume a STEM candidate.
Personalize to your clock →H-1B wage level
the lottery is wage-weighted — each level is one more entry
This range already reaches Level IV — the maximum four lottery entries.
DOL prevailing wage, 2026-27 wage year · Software Developers (15-1252) · San Francisco-Oakland-Fremont, CA. Wage level is derived by USCIS from the offered wage, occupation and worksite; the occupation shown is inferred from the job title.
Employer immigration record
from this employer's Department of Labor filings
Files H-1B transfers
Sourced from Department of Labor LCA, PERM and prevailing-wage disclosure data. Employer matching is by name, so figures may be split across an employer's legal entities. Absence of a filing means none appears in our copy of the data, not that none exists.
Community outcomes
No reports yet — be the first to help the next applicant.
About the role
- This is a systems-heavy, production-first role.
Responsibilities
- Design, build, and operate distributed infrastructure for large-scale model serving, including request routing, load balancing, batching, and multi-node coordination
- Optimize inference latency and throughput in production, including work on KV cache management, continuous batching, speculative decoding, and quantization
- Build and maintain high-concurrency serving systems with strong uptime, low tail latency, and deep observability
- Benchmark, tune, and extend inference engines to support new model architectures as they move from research into production
- Partner with research and infrastructure teams to translate emerging model designs into production-ready serving systems
- Build tooling for tracing, debugging, and resolving issues across the serving stack, from orchestration down to GPU kernels
- Participate in on-call rotation to support production inference systems
Requirements
- 3+ years of experience building and operating distributed systems in production
- Experience with production infrastructure at scale: reliability, observability, and performance under real-world load
- Strong systems programming skills in Python, C++, Rust, or similar languages
Nice to have
- Experience with LLM inference engines such as vLLM, SGLang, or TensorRT-LLM
- Familiarity with GPU programming (CUDA) or low-level performance optimization
- Experience with model parallelism, tensor/pipeline parallelism, or other distributed inference techniques
- Experience with Kubernetes or similar orchestration systems for GPU workloads
- Contributions to open-source ML systems or inference infrastructure projects
- Track record of operating large-scale production systems with strict latency and uptime requirements
- Visa sponsorship: We sponsor visas.
- While we can't guarantee success for every candidate or role, if you're the right fit, we're committed to working through the visa process together.
Compensation
- Depending on background, skills and experience, the expected annual salary range for this position is $350,000 - $500,000 USD (placeholder - verify against current internal bands before publishing).
Benefits
- Thinking Machines offers generous health, dental, and vision benefits, unlimited PTO, paid parental leave, and relocation support as needed.
Company info
- The mission of Thinking Machines is to build AI that extends human will and judgment.
- We are training frontier models with Inkling, developing Tinker to let people make models their own, and crafting interfaces that broaden human-AI communication.
Visa & Work Authorization
- We sponsor visas.
- While we can't guarantee success for every candidate or role, if you're the right fit, we're committed to working through the visa process together.
This listing is sourced directly from Thinking Machines Lab's careers page and normalized into a canonical job model.