Vanguard
Senior AI Scientist
Toronto, Canada · Senior
Sponsorship not specifiedDetected 5 days ago
PythonMachine LearningPyTorchscikit-learnData EngineeringData ScienceLLMsAgentic AIStatisticsA/B TestingResearchExperimental DesignCommunicationCollaboration
About the role
- This role sits at the intersection of AI safety research, LLM evaluation, and applied data science.
- You will also mentor junior data scientists and contribute to the growth of our AI safety research practice.
Responsibilities
- Design and execute adversarial evaluations of generative AI and agentic systems, probing for safety failures, policy violations, and unexpected behaviors under realistic conditions
- Apply rigorous experimental design and statistical methodology to AI safety research questions, including evaluation of model robustness and the reliability of automated evaluation systems
- Develop and validate quantitative frameworks for assessing LLM outputs and measuring evaluation quality at scale
- Build and maintain data pipelines to process large-scale evaluation outputs, aggregate metrics, and surface trends for research and stakeholder consumption
- Mentor and develop junior data scientists and analysts on the team
- Participate in special projects and perform other duties as assigned
Requirements
- Minimum 5 years of experience in data science, applied ML research, or AI evaluation roles
- Hands-on experience with large language models and agentic AI systems, with working knowledge of LLM behavior, failure modes, and safety evaluation techniques
- Familiarity with adversarial AI concepts including jailbreaks, prompt injection, and model robustness, whether through direct research or applied work
- Strong programming skills in Python, including experience with data pipelines, large-scale experimentation, and ML libraries such as PyTorch, Hugging Face, or Scikit-learn
Nice to have
- Experience in financial services, AI safety, trust and safety, or a regulated industry is a strong plus
- Clear, confident communication skills with the ability to explain technical findings to both research peers and non-technical stakeholders
- Bachelor's degree in Computer Science, Statistics, Applied Mathematics, or a related quantitative field
- Master's degree or equivalent research experience preferred
Company info
- identifying vulnerabilities in generative AI applications and ensuring that AI systems deployed at Vanguard behave safely, reliably, and in accordance with regulatory and business standards.
- We are looking for technically sharp, curious people who think rigorously about how AI systems fail and can communicate those findings clearly to the teams who need to act on them.
Apply directly at Vanguard →Create a free account for alerts like thisView Vanguard immigration profile
This listing is sourced directly from Vanguard's careers page and normalized into a canonical job model.