SumerSports
LLMOps Platform Engineer
Remote (United States)
Sponsorship not specifiedDetected 274 days ago
PythonFull-Stack DevelopmentVector DatabasesAWSGCPAzureCI/CDPlatform EngineeringMachine LearningDeep LearningData EngineeringNLPLLMsRAGLLMOpsA/B TestingResearch
About the role
- SumerSports is a leading football intelligence technology company that specializes in providing an innovative suite of products for football fans and NFL clubs.
- What sets us apart is our unique blend of big tech talent, data scientists, and former NFL personnel, who have a combined 600+ years of NFL experience.
- You'll work closely with AI app engineers, full stack engineers, and the Deep Learning Research group to ensure every AI system we ship is fast, grounded, and reliable.
Responsibilities
- Build and operate the LLM Platform: Develop model routing, prompt registry, and orchestration services for multi-model workflows.
- Enable fast, safe experimentation: Implement automated evaluation pipelines (offline + online) with golden sets, rubrics, and regression detection.
- Support CI/CD for prompt and model changes, with rollback and approval gates.
- Collaborate cross-functionally: Partner with product pods to instrument RAG pipelines and prompt versioning.
- Optimize performance and cost: Profile latency, token usage, and caching strategies.
- Build observability and monitoring for LLM calls, embeddings, and agent behaviors.
- Ensure reliability and safety: Implement guardrails (toxicity, PII filters, jailbreak detection).
- Maintain policy enforcement and audit logging for AI usage.
- Build and operate the
- Develop model routing, prompt registry, and orchestration services for multi-model workflows.
Requirements
- 5+ years of experience in applied ML, NLP, or ML infrastructure engineering.
- Strong coding skills in Python and experience with frameworks like LangChain, LlamaIndex, or Haystack.
- Experience deploying models or AI systems in production environments (AWS, GCP, or Azure).
- Familiarity with prompt management, LLM observability, and CI/CD automation for AI workflows.
Nice to have
- Experience with model serving (vLLM, Triton, Ray Serve, KServe).
- Understanding of LLM evaluation frameworks (OpenAI Evals, Promptfoo, Arize Phoenix, TruLens).
- Background in sports analytics, data engineering, or multimodal (video/text) systems.
- Exposure to Responsible AI practices (guardrails, safety evals, fairness testing).
Skills
- Our data-driven platform empowers teams with insights and tools to make informed decisions within salary cap constraints.
- The platform also serves the NCAA, offering insights around the transfer portal and more.
Benefits
- Work with deep learning and data teams to integrate structured and unstructured retrieval into LLM workflows.
Apply directly at SumerSports →Create a free account for alerts like thisView SumerSports immigration profile
This listing is sourced directly from SumerSports's careers page and normalized into a canonical job model.