Baseten

Baseten

Software Engineer - Inference Platform

San Francisco · Mid

Sponsorship not specifiedDetected 11 days ago
Distributed SystemsBackend DevelopmentKubernetesCI/CDMachine LearningSparkLLMsResearchCommunicationCollaboration

Stay score

odds of building a lasting career here

40Risky
Cap-exempt (no lottery)0
Sponsors this role90
Entry-level history0
PERM / green-card track0
Lottery odds40
Fits your clock70

Thin sponsorship signal and lottery-bound. A low-probability bet with your clock running. Prioritize cap-exempt roles and proven entry-level sponsors first.

Lottery odds assume a STEM candidate.

Personalize to your clock →

Community outcomes

No reports yet — be the first to help the next applicant.

About the role

  • The Baseten Inference Stack at NVIDIA Dynamo Day https://www.baseten.co/blog/nvidia-dynamo-day-baseten-inference-stack/
  • 2x faster inference with KV cache-aware routing https://www.baseten.co/blog/how-baseten-achieved-2x-faster-inference-with-nvidia-dynamo/
  • How Baseten multi-cloud capacity management (MCM) unifies deployments https://www.baseten.co/blog/how-baseten-multi-cloud-capacity-management-mcm-powers-cloud-self-hosted-and-hybr/#comparing-deployment-options-cloud-vs-self-hosted-vs-hybrid

Responsibilities

  • Build the infrastructure and orchestration systems that deploy and run large-scale distributed LLM inference, including routing, autoscaling, scheduling, and runtime management.
  • Design, build, and operate
  • Implement platform fundamentals such as API versioning, validation, usage metering, quotas, and authentication.
  • Instrument deep observability (metrics, traces, logs) and build repeatable benchmarks for speed, reliability, and quality. Help set best practices for testing, release automation, and operational excellence.
  • Own projects end to end, from architecture through deployment, monitoring, and iteration on customer feedback. Along the way, make thoughtful tradeoffs between performance, reliability, operational simplicity, and developer experience.
  • Design, build, and operate Model APIs, with a focus on advanced inference capabilities: structured outputs (JSON mode, grammar-constrained generation), tool/function calling, and multimodal serving.
  • We're growing quickly and recently raised our $1.5B Series F https://www.baseten.co/blog/announcing-our-series-f/, led by Altimeter Capital, Conviction Partners, and Spark Capital.
  • Join us and help build the platform engineers turn to ship AI products.

Requirements

  • Bachelor's, Master's, or Ph.D. in Computer Science, Engineering, or a related field, or equivalent practical experience.
  • A proven track record of owning low-latency, reliable backend services, including rate limiting, auth, quotas, metering, and migrations.
  • Comfort debugging performance and reliability issues across multiple layers of the stack, from application behavior down to runtime and infrastructure internals.

Nice to have

  • Experience with or contributions to LLM inference engines and frameworks such as vLLM, SGLang, TensorRT-LLM, TGI, or Dynamo.
  • Deep Kubernetes experience, including operators and custom resources, plus familiarity with service meshes or API gateways.
  • Experience with distributed scheduling, autoscaling, or service orchestration.
  • Experience operating GPU workloads in production.
  • A background in developer-facing infrastructure or APIs, or contributions to open-source infrastructure or ML systems.
  • Familiarity with observability tooling, CI/CD systems, or release automation.
  • Eagerness to learn new languages, frameworks, and systems, and a real interest in inference engineering.
  • Prior ML or LLM experience isn't required, though experience with model serving or inference systems is a plus.

Compensation

  • Competitive compensation, including meaningful equity

Benefits

  • Competitive compensation, including meaningful equity
  • (U.S. only) 100% coverage of medical, dental, and vision insurance for employee and dependents
  • Flexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!)
  • Paid parental leave
  • Fertility and family-building stipend through Carrot
  • Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities.
  • If you are a motivated individual with a passion for machine learning and a desire to be part of a collaborative and forward-thinking team, we would love to hear from you.
  • By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable companies operating at the frontier of AI to bring cutting-edge models into production.

Company info

  • Our inference platform empowers customers to deploy and operate cutting-edge models with industry-leading performance, scalability, and reliability.
  • At Baseten, we are committed to fostering a diverse and inclusive workplace.
  • Partner with Inference Performance engineers and other teams to make new optimizations broadly available to customers and easy to configure.
  • We're looking for distributed systems engineers and product-minded generalists to build the distributed runtime that powers large-scale LLM inference on Baseten.
Apply directly at Baseten →Create a free account for alerts like thisView Baseten immigration profile

This listing is sourced directly from Baseten's careers page and normalized into a canonical job model.