Cerebras Systems
Staff Software Engineer, Inference API
Sunnyvale, CA · Staff+
Sponsorship not specifiedDetected 28 days ago
PythonGoRustC++Cloud PlatformsKubernetesCI/CDLinuxAPI DevelopmentRESTgRPCMachine LearningPyTorchNLPLLMsMLOpsComplianceElectrical EngineeringCommunication
Stay score
odds of building a lasting career here
40Risky
Cap-exempt (no lottery)0
Sponsors this role90
Entry-level history0
PERM / green-card track0
Lottery odds40
Fits your clock70
Thin sponsorship signal and lottery-bound. A low-probability bet with your clock running. Prioritize cap-exempt roles and proven entry-level sponsors first.
Lottery odds assume a STEM candidate.
Personalize to your clock →Employer immigration record
from this employer's Department of Labor filings
Green-card filing pattern in this occupation
Context, not a finding about this posting: of this employer's 3 green-card filings in this occupation, 100% were for a worker who already held the job.
Files H-1B transfers
22 transfer filings in the last year, covering 22 workers. Median labor-condition decision: 7 days. An employer that already files transfers is one that can take over an existing H-1B.
Sourced from Department of Labor LCA, PERM and prevailing-wage disclosure data. Employer matching is by name, so figures may be split across an employer's legal entities. Absence of a filing means none appears in our copy of the data, not that none exists.
Community outcomes
No reports yet — be the first to help the next applicant.
About the role
- You will work closely with model enablement, compiler, runtime, cloud infrastructure, product, customer-facing teams, and customers directly.
- This is a hands-on software engineering role for someone who enjoys turning rapidly evolving ML capabilities into durable, production-quality APIs.
Responsibilities
- Build production ML inference APIs. Design, implement, and maintain APIs for chat completions, text generation, streaming, model configuration, tool calling, structured outputs, multimodal inputs, and other emerging inference capabilities.
- Deliver a unified serving experience. Create consistent request and response semantics across GPU prefill, Cerebras decode, and other heterogeneous inference backends.
- Own API compatibility and evolution. Maintain compatibility with widely adopted inference interfaces while designing Cerebras-specific extensions. Establish clear versioning, deprecation, validation, and backward compatibility practices.
- Improve serving performance. Optimize streaming behavior, time to first token, request latency, throughput, batching, serialization, tokenization, scheduling, and communication between serving components.
- Ensure functional and numerical correctness. Build validation systems for tokenization, sampling, logits, generated outputs, precision changes, model upgrades, determinism, and compatibility across serving backends.
- Develop testing and qualification infrastructure. Create conformance tests, workload-replay tools, model-validation suites, performance benchmarks, integration tests, and release gates.
- Collaborate across the stack. Partner with compiler, runtime, kernel, cloud, product, and solutions teams to translate model and customer requirements into scalable serving capabilities.
- Build production ML inference APIs.
- Design, implement, and maintain APIs for chat completions, text generation, streaming, model configuration, tool calling, structured outputs, multimodal inputs, and other emerging inference capabilities.
Requirements
- 5+ years of software engineering experience, including substantial individual-contributor ownership of production software or distributed systems.
- Experience integrating software across service, framework, runtime, and infrastructure boundaries.
- Experience designing or maintaining OpenAI-compatible, gRPC, REST, or streaming inference APIs.
- Experience with Linux, containers, Kubernetes or comparable orchestration systems, CI/CD, and operating latency-sensitive services in production.
- Ability to diagnose correctness, reliability, and performance issues across multiple components of a distributed serving system.
- Strong communication and cross-functional execution skills, with the ability to turn ambiguous model or product requirements into production-quality software.
- Bachelor's degree in computer science, Computer Engineering, Electrical Engineering, or a related discipline, or equivalent practical experience.
- Build the control and data paths required to coordinate GPU prefill with Cerebras decode, including request routing, state transfer, error handling, retries, and lifecycle management.
Nice to have
- Experience modifying or contributing to vLLM, SGLang, PyTorch, Hugging Face Transformers, Triton, TensorRT-LLM, or another open-source ML systems project.
- Experience creating API conformance, model-quality, numerical-comparison, determinism, or performance-regression test systems
- Experience with multi-model or multi-tenant inference platforms, including routing, admission control, fairness, quotas, rate limiting, and capacity-aware scheduling
- Understanding of model-specific tokenization, chat templates, generation configuration, logits processing, stopping criteria, tool calling, structured generation, and constrained decoding.
- Experience with disaggregated prefill/decode architectures, KV-cache transfer, prefix caching, chunked prefill, memory-aware admission control, or request scheduling.
- Familiarity with reduced-precision inference and quantization formats such as BF16, FP8, FP4, INT8, or INT4.
- Why Join Cerebras
- With dozens of model releases and rapid growth, we've reached an inflection point in our business.
Skills
- Cerebras works with the leading model labs, global enterprises, and cutting-edge AI-native startups.
Benefits
- Define end-to-end service indicators and build structured logging, tracing, metrics, dashboards, health checks, and diagnostic tooling for production inference traffic.
Company info
- Build a breakthrough AI platform beyond the constraints of the GPU.
- Work on one of the fastest AI supercomputers in the world.
- Enjoy job stability with startup vitality.
- Our simple, non-corporate work culture that respects individual beliefs.
- Improve developer experience. Build intuitive configuration, SDKs, documentation, examples, debugging tools, and self-service workflows for internal developers, customers, and partners.
Apply directly at Cerebras Systems →Create a free account for alerts like thisView Cerebras Systems immigration profile
This listing is sourced directly from Cerebras Systems's careers page and normalized into a canonical job model.