Anyscale
Distributed LLM Inference Engineer
San Francisco
Sponsorship not specifiedDetected 56 days ago
Distributed SystemsMachine LearningDeep LearningTensorFlowPyTorchData ScienceLLMsResearch
About the role
- As a Distributed LLM Inference Engineer, you will help systems and optimizations that push the boundaries of performance for inference at large scale.
- This is an incredibly critical role to Anyscale as it allows us to achieve a market leading position for AI infrastructure.
- Iterate very quickly with product teams to ship the end to end solutions for Batch and Online inference at high scale which will be used by open-source Ray users and customers of Anyscale
Requirements
- Familiarity with running ML inference at large scale with high throughput and low latency
- We'd love to hear from you if you have
Skills
- About Anyscale
- Contributions to deep learning frameworks (PyTorch, TensorFlow)
Compensation
- At Anyscale, we take a market-based approach to compensation.
Benefits
- This role is also eligible to participate in Anyscale's Equity and Benefits offerings, including the following:
- Education & Wellbeing Stipend
- Healthcare plans, with premiums covered by Anyscale at 99% for both employees and dependents
- 401k Retirement Plan
- Paid Parental Leave
- Fertility Benefits
- Commute reimbursement
Company info
- We are data-driven, transparent, and consistent.
Apply directly at Anyscale →Create a free account for alerts like thisView Anyscale immigration profile
This listing is sourced directly from Anyscale's careers page and normalized into a canonical job model.