Litellm

Senior Backend Engineer

San Francisco · Senior

Sponsorship not specifiedDetected 113 days ago

PythonFastAPIFlaskBackend DevelopmentPostgreSQLRedisPrometheusDatadogLLMsAuditing

About the role

You'll focus on owning our guardrails and logging world-class.
You will be in charge of the backend code that ensures all guardrail calls are consistently logged, errors are surfaced to users (not silently swallowed), and our observability instrumentation works for real-world, high-volume traffic.
Your attention to detail in areas like latency metrics, logging traceability, and backend guardrail registration will directly impact user trust in our security and compliance features.

Build and scale our product, ensuring performance, reliability, and continuous improvement.
Build and design CPU-level guardrails to cover common attacks on LLM API's / MCP servers / Agents
Work with observability integrations, including Datadog, Splunk, Prometheus, and OpenTelemetry, to maintain accurate, configurable, and usable monitoring and logging for backend systems
Collaborate cross-functionally on backend engineering priorities (performance, reliability, security)
Driven to deliver high-quality backend code with strong guardrails, auditing, and debugging capabilities

Bachelor's or Master's in Computer Science or related field
1+ years of experience with Python and backend frameworks (e.g. FastAPI, Flask)
Familiarity with database integration and troubleshooting (PostgreSQL, Redis, etc.)

What We're Looking For
Our platform empowers developers by providing secure, reliable access to LLMs and adjacent services, and we're looking for a Senior Backend Engineer to help us build rock-solid guardrails and observability tooling at scale.

This listing is sourced directly from Litellm's careers page and normalized into a canonical job model.