Anyscale
Distributed LLM Inference Engineer
San Francisco
Stay score
odds of building a lasting career here
Thin sponsorship signal and lottery-bound. A low-probability bet with your clock running. Prioritize cap-exempt roles and proven entry-level sponsors first.
Lottery odds assume a STEM candidate.
Personalize to your clock →Employer immigration record
from this employer's Department of Labor filings
Files H-1B transfers
Sourced from Department of Labor LCA, PERM and prevailing-wage disclosure data. Employer matching is by name, so figures may be split across an employer's legal entities. Absence of a filing means none appears in our copy of the data, not that none exists.
Community outcomes
No reports yet — be the first to help the next applicant.
About the role
- As a Distributed LLM Inference Engineer, you will help systems and optimizations that push the boundaries of performance for inference at large scale.
- This is an incredibly critical role to Anyscale as it allows us to achieve a market leading position for AI infrastructure.
- Iterate very quickly with product teams to ship the end to end solutions for Batch and Online inference at high scale which will be used by open-source Ray users and customers of Anyscale
Requirements
- Familiarity with running ML inference at large scale with high throughput and low latency
- We'd love to hear from you if you have
Skills
- About Anyscale
- Proud to be backed by Andreessen Horowitz, NEA, and Addition with $250+ million raised to date.
- Contributions to deep learning frameworks (PyTorch, TensorFlow)
Compensation
- At Anyscale, we take a market-based approach to compensation.
Benefits
- This role is also eligible to participate in Anyscale's Equity and Benefits offerings, including the following:
- Healthcare plans, with premiums covered by Anyscale at 99% for both employees and dependents
- Education & Wellbeing Stipend
- 401k Retirement Plan
- Paid Parental Leave
- Fertility Benefits
- Commute reimbursement
Company info
- About Anyscale At Anyscale, we're on a mission to democratize distributed computing and make it accessible to software developers of all skill levels.
- We are data-driven, transparent, and consistent.
This listing is sourced directly from Anyscale's careers page and normalized into a canonical job model.