Perplexity AI
Member of Technical Staff (Model Behavior Architect)
San Francisco · Staff+
Sponsorship not specifiedDetected 71 days ago
PythonLLMsA/B TestingResearchCommunicationCollaboration
About the role
- This role is equal parts craft and science.
- You'll serve as a go-to expert on prompting, model quality, and behavioral consistency across new product features and model releases.
Responsibilities
- Context Engineering: Design, test, and optimize context strategies and system prompts that shape answer engine behavior across products, features, and use cases.
- Evaluation Systems: Build automated and semi-automated evaluation pipelines that measure model quality, catch regressions, and scale across product surfaces.
- Model Launch Support: Partner with research and engineering to validate model behavior before and during rollouts, ensuring smooth transitions with no degradation.
- Research & Analysis: Identify inconsistencies and failure modes in model outputs through well-designed research projects - for both internal and production-facing systems.
- Cross-functional Collaboration: Work closely with design, product, and research teams to translate product goals into concrete model behavior requirements.
- Knowledge Sharing: Help engineers across teams build intuition for prompt design, context engineering, and evaluation best practices.
- Ability to manage multiple concurrent projects in a fast-moving environment.
- Design, test, and optimize context strategies and system prompts that shape answer engine behavior across products, features, and use cases.
- Build automated and semi-automated evaluation pipelines that measure model quality, catch regressions, and scale across product surfaces.
Requirements
- Experience designing evaluations, benchmarks, or metrics for AI systems.
- Strong experience with Perplexity or other frontier AI models in production settings.
- Demonstrated experience with Python - you'll prototype, debug, automate, and build systems at scale.
- 3+ years of experience working with LLMs in a product or research setting.
Nice to have
- Experience with A/B testing or experimentation frameworks.
- Track record of improving AI system performance through systematic evaluation and iteration.
Apply directly at Perplexity AI →Create a free account for alerts like thisView Perplexity AI immigration profile
This listing is sourced directly from Perplexity AI's careers page and normalized into a canonical job model.