Drata
Applied Research Engineer
Hybrid - San Francisco · Exec
Sponsorship not specified$145k-$196kDetected 4 days ago
PythonVector DatabasesRESTMachine LearningData ScienceNLPLLMsRAGAgentic AIA/B TestingResearchCollaboration
About the role
- This is a research-focused role emphasizing experimentation and rigor over production engineering.
- Explore ML + probabilistic approaches where GenAI is not the best fit: classifiers, ranking models, graph/link prediction, calibration, and structured prediction
- Stay current on applied research in RAG, agents, LLM evaluation, and relevance modeling; bring innovations into the product
Responsibilities
- Build and maintain evaluation frameworks: golden datasets, automated quality metrics, regression detection
- Run experiments to validate hypotheses and quantify improvements before production rollout
- Debug failure modes and build error taxonomies across retrieval, reasoning, and generation
- Collaborate with AI and Software Engineers to hand off validated approaches for productionization
- 1+ years of hands-on experience building or contributing to production AI/ML systems
Requirements
- 3+ years of experience in applied research, data science, or ML with a focus on NLP, information retrieval, or knowledge systems
- Experience communicating research findings to engineering teams and translating insights into actionable improvements
- Experience with RAG systems: chunking strategies, vector databases, retrieval optimization
- Proficiency in evaluation methodology: metrics design, golden dataset creation, A/B testing, statistical significance
Nice to have
- Experience with compliance, legal, or document-heavy domains
- Publications or contributions in IR, NLP, or RAG evaluation
Skills
- golden datasets, automated quality metrics, regression detection
- Strong Python skills and comfort with notebook-driven research workflows
Compensation
- A variety of factors are considered when determining someone's leveling and compensation-including a candidate's professional background and experience.
- This role will receive a competitive base salary, benefits, and stock, typically in the form of Restricted Stock Units (RSUs).
- $145,200 - $196,400.
Benefits
- We provide stock equity to ensure that as the company grows, you share directly in that success.
- Equity gives every employee a sense of ownership and the opportunity to celebrate our wins together-because your contributions don't just support our progress; they help drive our collective success.
- We want to support you in life's most important moments, so we offer a paid Parental Leave policy, after six months of employment.
- Employees also receive access to Kindbody fertility and family-building benefits and dedicated leave specialists who help guide you through the entire process.
- Bonus: Experience with compliance, legal, or document-heavy domains
- Bonus: Publications or contributions in IR, NLP, or RAG evaluation
- Implement and tune ranking/reranking systems: cross-encoders, LLM-based rerankers, learning-to-rank, custom scoring functions
- Drata offers a flexible vacation policy, paid holidays, and other perks to recharge.
Company info
- Hear the Voice of the Team https://drata.com/about/life-at-drata: Explore our "Life at Drata" page for employee testimonials on our collaborative and the growth opportunities available.
- Experience the Impact https://www.greatplacetowork.com/certified-company/7044563: See why we are consistently recognized on Fortune's Best Workplaces lists.
- LinkedIn https://www.linkedin.com/company/drata/posts/?feedView=all - follow us for company updates, employee stories, and career news.
- A comprehensive suite of financial benefits, including a 401(k) plan, company-paid life and disability insurance, tax-advantaged spending accounts, and a range of discounted voluntary offerings to help you customize and strengthen your overall financial position.
This listing is sourced directly from Drata's careers page and normalized into a canonical job model.