SpreeAI
Software Engineer (Model Evaluation & Benchmarking)
Hybrid (San Francisco, California, US)
Sponsorship not specifiedDetected 20 days ago
PythonJavaC++Object-Oriented ProgrammingData StructuresAlgorithmsCI/CDMachine LearningPandasNumPyData AnalysisLLMsA/B TestingResearch
About the role
- This position focuses on evaluating generative and vision-based models through automated benchmarking, dataset-driven testing, and performance validation pipelines.
- You will work at the intersection of applied science, infrastructure, and product - helping define how we measure realism, consistency, and quality across image, video, and multimodal AI systems.
- Why This Role Exists Modern AI evaluation extends beyond pass/fail testing.
Responsibilities
- Build automated evaluation pipelines for multimodal AI models.
- Validate model checkpoints and detect regressions across versions.
- Develop evaluation metrics for realism, consistency, and performance.
- Collaborate with ML researchers and infrastructure teams to ensure production readiness.
- We thrive in a dynamic, fast-paced environment where creativity meets technology to drive real impact.
- About the Role We are hiring Engineers focused on AI Model Evaluation to build the systems that ensure multimodal AI behaves reliably, consistently, and predictably as it moves from research into production.
- We are building evaluation systems where research velocity and product reliability must coexist.
Nice to have
- Familiarity with HuggingFace ecosystem or open-source ML toolkits.
- Knowledge of diffusion models or multimodal architectures.
- Experience with data analysis tools (NumPy, Pandas, visualization libraries).
Benefits
- Benchmark diffusion models, vision systems, and generative workflows.
- Understanding of machine learning experimentation workflows.
Company info
- We are hiring Engineers focused on AI Model Evaluation to build the systems that ensure multimodal AI behaves reliably, consistently, and predictably as it moves from research into production.
- Our mission is to redefine the retail landscape with cutting-edge AI solutions that blend high fashion and technology.
Apply directly at SpreeAI →Create a free account for alerts like thisView SpreeAI immigration profile
This listing is sourced directly from SpreeAI's careers page and normalized into a canonical job model.