Rhoda AI

Inference Infrastructure Engineer

Mountain View

Sponsorship not specifiedDetected 70 days ago

Distributed SystemsAWSGCPKubernetesgRPCKafkaMachine LearningPyTorchMLOpsRoboticsHardware DesignResearch

About the role

We've raised over $450M and are investing aggressively in model research, infrastructure, hardware development, and manufacturing scale-up to make generalist robotics a reality.
You'll be responsible for running large foundation models efficiently and reliably across cloud and on-prem environments, with a focus on resource management, scheduling, and infrastructure scalability.

Design and operate large-scale infrastructure to run model workloads across cloud and on-prem environments
Build and maintain Kubernetes-based deployment pipelines for managing distributed ML workloads
Own resource scheduling and orchestration across GPU clusters - optimizing utilization, workload balancing, and cost-performance tradeoffs
Integrate and manage ML frameworks and model serving systems (e.g., Triton, Ray Serve, TorchServe) across research and production use cases
Build tooling for model deployment, versioning, and observability to support fast iteration cycles

3+ years of experience in ML infrastructure, MLOps, or distributed systems
Strong proficiency with Kubernetes and containerized deployment pipelines
Experience with GPU orchestration and resource scheduling across large distributed jobs
Familiarity with ML frameworks (e.g., PyTorch, JAX) and model serving tools (e.g., Triton, Ray Serve, TorchServe)
Experience with streaming systems or high-throughput data transport (e.g., Kafka, gRPC, NATS)
Familiarity with on-robot or embedded inference environments
Experience with large-scale cluster topology and scheduling systems (e.g., SLURM, Ray, Volcano)

This listing is sourced directly from Rhoda AI's careers page and normalized into a canonical job model.