Rime
Software Engineer, ML Serving
United States
Sponsorship not specifiedDetected 34 days ago
Node.jsAWSGCPCloud PlatformsDockerKubernetesTerraformAnsibleCI/CDLinuxDevOpsSite Reliability EngineeringPlatform EngineeringRESTgRPCWebSocketsMachine LearningVoIPCollaboration
About the role
- As Rime moves toward its next-generation architecture, you'll be a core architect of how our models get served.
- Architecture and implementation of Rime's TTS serving infrastructure, from GPU-backed inference engines to the API surface.
- Model optimization from a single-node to disaggregated fleet serving.
Responsibilities
- build voice AI for human connection, not slop.
- We're hiring a Software Engineer to own the serving infrastructure that connects Rime's inference engines to the world.
Requirements
- Hands-on experience with real-time multinode ML serving infrastructure - ML serving framework experience: NVIDIA Dynamo/Triton, vLLM, SGLang, or equivalent.
- Experience with distributed or disaggregated model serving (Tensor Parallel, Pipeline Parallel, or equivalent).
- Hands-on experience with real-time multinode ML serving infrastructure -
Nice to have
- Experience with multinode training (DDP, FSDP, etc.).
- Experience with gRPC or other bidirectional binary streaming protocols.
- Experience with audio streaming and related technologies (WebRTC, WebSockets, etc.).
- Experience with a multilingual monorepo where you pick the best language out of merit more than personal experience.
- Experience with multi-cloud infrastructures (AWS, GCP, OCI, etc.).
- Comfort with configuration management tooling (Ansible, Chef, Puppet, or similar).
- SRE, DevOps, or platform engineering background at a startup.
- Experience at an early-stage company.
Skills
- Compatibility with different NVIDIA hardwares from Hopper to Blackwell and beyond for on-prem and cloud deployments.
Company info
- The systems you build determine what experiences our customers can deploy at scale.
- Meaningful equity upside at an early stage.
- SF / Bay Area.
- Are outliers
- Cut through the hype to focus on the craft
- Move fast with agency and freedom
- Maintain a growth mindset, finding joy in the struggle
- Do the right things, knowing that it'll lead to making money
- you can help us set the vision.
- AT RIME, WE...
- If that sounds like you too, you'll be a great fit for Rime!
- Rime is a foundation modeling company that builds voice AI for enterprises running customer experiences at scale.
- Our models are purpose-built for high-volume conversational deployments, engineered for the accuracy, performance, and deployment flexibility that production environments actually demand.
This listing is sourced directly from Rime's careers page and normalized into a canonical job model.