Modular

Modular

Cloud Inference Engineer

USA

Sponsorship not specified$167k-$273kDetected 187 days ago
GoNode.jsDistributed SystemsKubernetesHelmMachine LearningLLMs

About the role

  • We're seeking engineers who are passionate about pushing the boundaries of distributed inference systems and enjoy working at the intersection of large-scale systems and machine learning.
  • We are looking for candidates based on their breadth and depth of experience in backend engineering, AI inference, and distributed systems development.

Responsibilities

  • Build & ship a LLM focused inference platform using best in class inference techniques (disaggregated inference, multi-node deployment of large models, high performance networking, distributed kv-cache management, high throughput batch processing, etc) Push the envelope for operational excellence with request-to-kernel observability, multi-cloud deployments, clever autoscaling, cold-start optimizations, and more.
  • Collaborate with our kernels and genAI teams to achieve SOTA application performance by integrating SOTA kernel & serving optimizations with SOTA cluster optimizations.
  • Build helm charts, kubernetes operators, and more to make a create simple, effective, maintainable deployments.
  • Team Building Events.

Requirements

  • Experience with high performance computing / networking Experience working on high scale ML inference infrastructure (traditional

Compensation

  • The estimated base salary range for this role to be performed in the US, regardless of the state, is $166,500.00 - $273,000.00 USD.
  • The estimated base salary range for this role to be performed in Canada, regardless of the province, is $158,000.00 - $258,000.00 CAD.

Benefits

  • World-class Benefits.
  • Premier insurance plans, up to 5% 401k matching, flexible paid time off, and more are available to you!
  • We offer very strong compensation packages, including stock options.
  • The total compensation for a candidate will also include annual target bonus, equity, and benefits, with equity making up a significant portion of your total compensation.
  • Please note that specific benefit packages may vary based on your location.

Company info

  • We are a progressive and agile team with some of the industry's best engineering and product leaders.
Apply directly at Modular →Create a free account for alerts like thisView Modular immigration profile

This listing is sourced directly from Modular's careers page and normalized into a canonical job model.