Steampunk, Inc.
AI Evaluation Scientist
McLean, VA, US
Sponsorship not specified$105k-$145kDetected 1 day ago
PythonAWSAzureMachine LearningPyTorchscikit-learnData ScienceNLPLLMsRAGLLMOpsStatisticsA/B TestingResearchExperimental DesignCommunication
About the role
- Overview We are looking for an AI Evaluation Scientist to design and execute evaluation processes that ensure our predictive and generative AI systems are accurate, reliable, safe, and aligned with mission requirements. This role is essential for establishing trust in AI solutions and supporting continuous improvement across the AI lifecycle. The AI
- Evaluation Scientist will work closely with engineers, data scientists, governance analysts, and product teams to develop evaluation metrics, build test harnesses, analyze model behavior, and support responsible deployment. Contributions Responsibilities include: Implement evaluation frameworks for AI models, including accuracy, robustness, relevance, bias,
Nice to have
- This role is essential for establishing trust in AI solutions and supporting continuous improvement across the AI lifecycle.
- Contributions Responsibilities include:
- Assist in mapping evaluation outcomes to responsible AI principles such as fairness, transparency, reliability, and safety.
- You will contribute to the growth of our AI & Data Exploitation Practice!
- Ability to hold a position of public trust with the U.S. government.
- Proficiency in Python and relevant libraries such as PyTorch, Hugging Face, scikit-learn, LangChain.
- Proficiency in AI evaluation frameworks such as Ragas.
- Understanding of LLM behavior, prompt evaluation, retrieval pipelines, or RAG architectures.
Skills
- Stay current with emerging evaluation tools, frameworks, metrics, and research related to LLM assessment and generative AI reliability.
Compensation
- The estimate displayed represents a typical annual salary range for this position.
- Annual salary is just one aspect of Steampunk's total compensation package for employees.
Benefits
- Learn more about additional Steampunk benefits here.
- Steampunk is a Change Agent in the Federal contracting industry, bringing new thinking to clients in the Homeland, Federal Civilian, Health and DoD sectors.
Company info
- Identity Statement As part of the
- Through our Human-Centered delivery methodology, we are fundamentally changing the expectations our Federal clients have for true shared accountability in solving their toughest mission challenges.
Apply directly at Steampunk, Inc. →Create a free account for alerts like thisView Steampunk, Inc. immigration profile
This listing is sourced directly from Steampunk, Inc.'s careers page and normalized into a canonical job model.