Lumalabs Ai
Qualitative Evaluation Engineer
Redwood City, USA
Sponsorship not specifiedDetected 367 days ago
LLMsUX ResearchResearchCommunication
About the role
- We're seeking a candidate to help shape and scale the way we understand, measure, and improve model performance.
- Evaluate generative model performance across diverse tasks, prompts, and modalities.
- Identify key failure modes, regression patterns, and edge cases that impact product quality.
Responsibilities
- Collaborate closely with technical artists and engineers to align evaluations with model capabilities and target use cases.
- About Luma AI Luma's mission is to build multimodal AI to expand human imagination and capabilities.
- About the Role Luma is pushing the boundaries of generative AI, building tools that redefine how visual content is created.
- This is not a checkbox metrics role - it's about building evaluative systems that match the complexity of human perception, creativity, and intention.
- Develop and maintain qualitative evaluation frameworks that are scalable and reusable.
- Lead qualitative studies, side-by-side comparisons, and human-in-the-loop evaluation efforts.
Requirements
- Master's degree or higher in Cognitive Science, Human-Computer Interaction (HCI), Design Research, Psychology, Media Studies, or a related field.
- 5+ years of experience in product evaluation, UX research, model testing, or similar roles that involve structured qualitative assessment.
- Strong systems thinking and the ability to define abstract qualities (like believability, identity retention, or scene coherence) in clear evaluative terms.
- Excellent written communication skills and the ability to synthesize nuanced judgments into clear, actionable insights.
- Qualifications Master's degree or higher in Cognitive Science, Human-Computer Interaction (HCI), Design Research, Psychology, Media Studies, or a related field.
Skills
- Stay informed about emerging evaluation standards in generative AI and creative tools.
Benefits
- To go beyond language models and build more aware, capable, and useful systems, the next step function change will come from vision.
Company info
- So, we are working on training and scaling up multimodal foundation models for systems that can see and understand, show and explain, and eventually interact with our world to effect change.
Apply directly at Lumalabs Ai →Create a free account for alerts like thisView Lumalabs Ai immigration profile
This listing is sourced directly from Lumalabs Ai's careers page and normalized into a canonical job model.