Baseten

Baseten

Engineering Manager - Inference Performance

San Francisco

Sponsorship not specified$50k-$90kDetected 10 days ago
Machine LearningPyTorchSparkLLMsSystems EngineeringResearchLeadershipCommunicationMentoring

Stay score

odds of building a lasting career here

39Risky
Cap-exempt (no lottery)0
Sponsors this role90
Entry-level history0
PERM / green-card track0
Lottery odds (Level I)39
Fits your clock70

Thin sponsorship signal and lottery-bound (~15% per draw). A low-probability bet with your clock running. Prioritize cap-exempt roles and proven entry-level sponsors first.

Lottery odds assume a STEM candidate.

Personalize to your clock →

Community outcomes

No reports yet — be the first to help the next applicant.

About the role

  • This team makes the world's most demanding AI workloads run faster and more efficiently on GPUs.
  • This is a hands-on technical leadership role.
  • Your team's output directly affects how fast our customers' models run and how efficiently we serve them.

Responsibilities

  • Lead, mentor and grow a team of inference performance engineers through regular 1:1s, clear feedback, career development and performance reviews.
  • Hire top GPU and inference engineering talent, and build a strong, collaborative team culture as the runtime team scales.
  • Own the technical roadmap and execution for runtime performance work, balancing customer needs, new model launches and long-term platform investments.
  • Stay close to the technical work. Review designs, guide profiling and optimization efforts, and help the team reason from first principles about where time and memory go.
  • Drive the productionization of inference techniques such as quantization, speculative decoding, KV-cache reuse, chunked prefill and custom scheduling.
  • Partner with Infrastructure, Inference Platform, Kernels, Model APIs and customer-facing teams to set priorities, coordinate launches and ship wins.
  • We're growing quickly and recently raised our $1.5B Series F https://www.baseten.co/blog/announcing-our-series-f/, led by Altimeter Capital, Conviction Partners, and Spark Capital.
  • Join us and help build the platform engineers turn to ship AI products.
  • You'll manage and grow a team of inference performance engineers working across the inference engine and runtime: kernels, scheduling, batching, KV-cache management, speculative decoding and prefill/decode disaggregation.

Requirements

  • Bachelor's, Master's, or Ph.D. degree in Computer Science, Engineering, Mathematics, or a related field.
  • Experience managing engineers, including hiring, mentoring, giving feedback and running performance reviews.
  • Experience leading or closely supporting GPU optimization teams in training, inference or recommendation systems.
  • Familiarity with ML libraries such as PyTorch, TensorRT or TensorRT-LLM.
  • A track record of driving roadmaps and shipping complex technical projects with a team.
  • Clear written and verbal communication, including the ability to align stakeholders across teams.

Nice to have

  • Familiarity with inference engines such as vLLM, SGLang or TensorRT-LLM.
  • Experience with LLM optimization techniques (e.g., quantization, speculative decoding, continuous batching) in production.
  • Experience with GPU kernels (CUDA, Triton, CUTLASS, or similar).
  • Experience scaling a team through rapid growth at a startup.
  • A background as a hands-on performance or systems engineer before moving into management.

Compensation

  • Competitive compensation, including meaningful equity

Benefits

  • Competitive compensation, including meaningful equity
  • (U.S. only) 100% coverage of medical, dental, and vision insurance for employee and dependents
  • Flexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!)
  • Paid parental leave
  • Fertility and family-building stipend through Carrot
  • Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities.
  • If you are a motivated individual with a passion for machine learning and a desire to be part of a collaborative and forward-thinking team, we would love to hear from you.
  • By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable companies operating at the frontier of AI to bring cutting-edge models into production.

Company info

  • At Baseten, we are committed to fostering a diverse and inclusive workplace.
Apply directly at Baseten →Create a free account for alerts like thisView Baseten immigration profile

This listing is sourced directly from Baseten's careers page and normalized into a canonical job model.