Baseten

Baseten

Software Engineer - Inference Performance

San Francisco

Sponsorship not specifiedDetected 11 days ago
PythonMachine LearningPyTorchSparkLLMsLoad BalancingResearch

Stay score

odds of building a lasting career here

40Risky
Cap-exempt (no lottery)0
Sponsors this role90
Entry-level history0
PERM / green-card track0
Lottery odds40
Fits your clock70

Thin sponsorship signal and lottery-bound. A low-probability bet with your clock running. Prioritize cap-exempt roles and proven entry-level sponsors first.

Lottery odds assume a STEM candidate.

Personalize to your clock →

Community outcomes

No reports yet — be the first to help the next applicant.

About the role

  • Agentic Kernels in Production https://www.baseten.co/blog/agentic-kernels-in-production/
  • How we built the new fastest API for GLM-5.2 https://www.baseten.co/blog/how-we-built-the-new-fastest-api-for-glm-52/
  • Live draft model training for speculative decoding https://www.baseten.co/blog/live-draft-model-training-for-speculative-decoding/#eliminating-traditional-bottlenecks

Responsibilities

  • Implement and productionize cutting-edge inference techniques, working deep in runtime internals.
  • Profile and optimize inference end to end, from kernel launch overhead and memory layout up to request scheduling, prefill/decode disaggregation, and cache-aware routing.
  • Build benchmarking frameworks that measure real-world performance across model architectures, batch sizes, sequence lengths, and hardware configurations.
  • Contribute upstream to open-source inference engines (vLLM, SGLang, TensorRT-LLM), and partner closely with model, infrastructure, and customer-facing teams to ship wins.
  • We're growing quickly and recently raised our $1.5B Series F https://www.baseten.co/blog/announcing-our-series-f/, led by Altimeter Capital, Conviction Partners, and Spark Capital.
  • Join us and help build the platform engineers turn to ship AI products.

Requirements

  • Bachelor's, Master's, or Ph.D. degree in Computer Science, Engineering, Mathematics, or related field.
  • Experience with one or more general-purpose programming languages, such as Python or C++.
  • Familiarity with LLM optimization techniques (e.g., quantization, speculative decoding, continuous batching).
  • Demonstrated interest and experience in LLMs.

Nice to have

  • Proficiency in enhancing the performance of software systems, particularly in the context of large language models (LLMs)
  • Contributed to vLLM, SGLang, TensorRT-LLM, or another inference engine.
  • Worked on large-scale distributed serving: autoscaling, load balancing, multi-region or multi-cloud capacity.
  • Written or optimized GPU kernels (CUDA, Triton, CUTLASS, or similar)
  • Worked on quantization (FP8/FP4, AWQ, GPTQ) or speculative decoding in production.

Compensation

  • Competitive compensation, including meaningful equity

Benefits

  • Competitive compensation, including meaningful equity
  • (U.S. only) 100% coverage of medical, dental, and vision insurance for employee and dependents
  • Flexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!)
  • Paid parental leave
  • Fertility and family-building stipend through Carrot
  • Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities.
  • If you are a motivated individual with a passion for machine learning and a desire to be part of a collaborative and forward-thinking team, we would love to hear from you.
  • By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable companies operating at the frontier of AI to bring cutting-edge models into production.

Company info

  • THE ROLE We're looking for inference performance engineers who want to make the world's most demanding AI workloads run faster and more efficiently.
  • Your work directly impacts how fast our customers' models run and how efficiently we serve them.
  • Turn performance into cost savings. Improve tokens per GPU-hour, raise utilization, and give customers and internal teams clear latency/throughput/cost tradeoffs.
  • At Baseten, we are committed to fostering a diverse and inclusive workplace.
Apply directly at Baseten →Create a free account for alerts like thisView Baseten immigration profile

This listing is sourced directly from Baseten's careers page and normalized into a canonical job model.