SambaNova
Senior Software Engineer, ML Infrastructure
Remote - US · Senior · Full-time
Sponsorship not specified$200k-$275kDetected 14 days ago
PythonDistributed SystemsFull-Stack DevelopmentData StructuresAlgorithmsCode ReviewMachine LearningLLMsElectrical Engineering
About the role
- SambaNova is an inference-first company, and this role sits at the heart of that mission: turning state-of-the-art inference techniques into reliable, high-throughput, low-latency services exposed to customers through SambaStack and SambaCloud.
Responsibilities
- Design and productionize advanced inference techniques on RDU to optimize for performance and cost.
- The engineer will own end-to-end systems spanning request scheduling, advanced decoding algorithms, caching layers, API surfaces, and the accuracy infrastructure that keeps the stack trustworthy.
- This role partners closely with ML, compiler, runtime, and product teams to ship inference features from prototype to production.
Requirements
- Experience designing and maintaining production services with strict latency, throughput, and availability requirements
- Working knowledge of modern LLM inference techniques and familiarity with open-source serving stacks such as vLLM, TensorRT-LLM, or SGLang
- Proficiency in Python
- 5+ years of industry experience building and operating large-scale distributed systems, ideally in ML serving
- Strong software engineering fundamentals: algorithms, data structures, concurrency, and systems design
- Experience collaborating across teams to deliver complex, system-level engineering solutions
- Design and productionize advanced inference techniques on RDU to optimize for performance and cost. Key areas include speculative decoding, constrained decoding, function/tool calling, prompt caching, and long-context inference.
- Own SambaNova's integration with vLLM and adjacent serving frameworks, adapting them to RDU's architecture.
- Own the public inference API surface exposed through SambaStack and SambaCloud.
- Build and maintain the accuracy verification and regression infrastructure that gates every inference feature shipped to customers.
- Partner with ML, compiler, runtime, and product teams to take inference features from prototype to production.
- Contribute to technical design discussions, code reviews, and architectural decisions as a senior individual contributor.
Nice to have
- Bachelor's degree in Computer Science, Electrical Engineering, or related field
Compensation
- Base Pay Range
- $200,000 - $275,000 USD
- SambaNova offers a competitive total rewards package, including the base salary, plus equity and benefits.
Benefits
- SambaNova offers a competitive total rewards package, including the base salary, plus equity and benefits.
- We cover 95% premium coverage for employee medical insurance, and 77% premium coverage for dependents and offer a Health Savings Account (HSA) with employer contribution.
- We also offer Dental, Vision, Short/Long term Disability, Basic Life, Voluntary Life, and AD&D insurance plans in addition to Flexible Spending Account (FSA) options like Health Care, Limited Purpose, and Dependent Care.
- Benefits Summary for US-Based, Full-Time Employment Positions
Equal opportunity
- SambaNova Systems is an Equal Opportunity/Affirmative Action Employer.
Apply directly at SambaNova →Create a free account for alerts like thisView SambaNova immigration profile
This listing is sourced directly from SambaNova's careers page and normalized into a canonical job model.