RxSense

RxSense

Principal AI Ops Engineer

Remote (US) · Principal

Sponsorship not specified$190k-$225kDetected 9 days ago
TypeScriptPython.NETCode ReviewAWSKubernetesCI/CDDevOpsMachine LearningLLMsLangGraphAgileAccountingLeadershipCommunicationCollaborationProblem Solving

About the role

  • Stand up and operate the runtime and lifecycle infrastructure for production agents, including deployment, versioning, monitoring, rate-limiting, and retirement.
  • Define the deployment contract (config, secrets, tools, memory, evals) and the operational SLOs.
  • Stand up self-hosted inference for workloads where managed APIs aren't the right fit, including latency-sensitive paths, regulated data, cost optimization, and vendor redundancy.

Responsibilities

  • Build and maintain end-to-end deployment pipelines for AI-powered applications, including artifact builds, environment promotion, rollback, and observability hooks.
  • Drive new greenfield deployment platforms from initial build to the default that AI teams ship on.
  • Own how the organization provisions, rotates, scopes, and meters access to model provider APIs (Anthropic, OpenAI, and others).
  • Build a key management layer that enforces per-team and per-app quotas, prevents leakage, and gives finance and engineering a clear view of spend.
  • Build evals into the CI/CD pipeline so no agent or LLM-powered service ships without passing a defined eval bar. Design the framework so product teams can author their own evals against a shared harness, and so eval results gate promotion across environments.
  • Own the serving stack, the autoscaling and GPU economics behind it, and the playbook for when a workload belongs to a managed provider versus internal infrastructure.
  • Design and build the shared developer harness that every AI-powered service uses: prompt management, model routing, retries, tracing, eval hooks, and policy enforcement.
  • Partner with finance on cost visibility, including token accounting, per-feature cost attribution, and real-time spend observability.
  • 6+ years of platform, infrastructure, or DevOps engineering, with at least 2 years building production infrastructure for AI/ML or LLM-powered systems. We care more about depth and drive than years on a resume.

Requirements

  • Production experience with LLM-powered applications, including prompt management, model routing, retries, tracing, and the operational realities of running agents or chains in production.

Nice to have

  • Direct experience integrating with Anthropic, OpenAI, or other frontier model provider APIs at scale, including key management, quota enforcement, and capacity planning.
  • Hands-on experience self-hosting models with vLLM, TGI, SGLang, or similar inference servers, including GPU autoscaling and cost optimization.
  • Built or contributed to an eval framework that gated production deployments.
  • Familiarity with agent runtimes and frameworks such as the Claude Agent SDK, LangGraph, or in-house equivalents.
  • Working familiarity with.NET, enough to read code, debug a deploy, and pair with service owners.
  • FinOps experience, particularly attributing AI spend to features or business units.
  • Kubernetes operator experience or comfort with custom controllers.
  • Experience with Agile development methodologies, preferably both Scrum and Kanban

Skills

  • Proven analytical thinking and problem-solving skills.
  • Excellent communication skills, both verbal and written.

Compensation

  • $190,000 - $225,000

Benefits

  • Participate in code review and promote collaboration and best practices including simplicity, automation, sound design patterns, test coverage, and reusability.
  • Education/Experience/Competencies:

Company info

  • We are a healthcare technology company that provides platforms and solutions to improve the management and access of cost-effective pharmacy benefits.
  • Our technology helps enterprise and partnership clients simplify their businesses and helps consumers save on prescriptions.
  • As a leader in SaaS technology for healthcare, we offer innovative solutions with integrated intelligence on a single enterprise platform that connects the pharmacy ecosystem.
  • With our expertise and modern, modular platform, our partners use real-time data to transform their business performance and optimize their innovative models in the marketplace.

Equal opportunity

  • RxSense believes that a diverse workforce is a more talented and productive workforce.
  • As such, we are an Equal Opportunity and Affirmative Action employer.
  • We believe in the strength of the collaboration, creativity and sense of community a diverse workforce brings.
  • Neither will qualified applicants be discriminated against on the basis of disability or protected veteran status.
Apply directly at RxSense →Create a free account for alerts like thisView RxSense immigration profile

This listing is sourced directly from RxSense's careers page and normalized into a canonical job model.