Senior Applied AI Scientist

New York, NY · Senior

Sponsorship not specified$182k-$220kDetected 25 days ago

PythonFull-Stack DevelopmentSQLMachine LearningData EngineeringData ScienceLLMsLLMOpsA/B TestingLogisticsResearchExperimental Design

About the role

This role sits at the intersection of data science, applied machine learning, and product engineering.
You'll work closely with engineers and product teams to evaluate production systems, run experiments, identify failure modes, and ensure our AI products become more accurate, reliable, and cost-effective over time.

Design and own evaluation frameworks for production LLM features, including LLM-as-a-judge evaluations, regression suites, synthetic datasets, golden datasets, and human review workflows.
Design and run experiments including prompt variations, workflow changes, retrieval improvements, and model comparisons
Define the metrics that matter and build dashboards that make AI performance visible across the organization.
Partner with engineering to determine which optimizations should be productionized and how to measure ongoing success.
Mentor teammates on experimental design, statistical rigor, evaluation methodology, and measurement best practices.
We're building a new system.
One designed from scratch for the digital age.
At Ro, technology isn't just a function… It's core to how we deliver care.

Strong Python and SQL skills with experience working on production data pipelines and experimentation.
You have experience designing reproducible evaluation frameworks rather than relying on manual spot checks or qualitative assessments.
You have strong statistical intuition: you think in terms of distributions, confidence intervals, variance, and sample sizes rather than anecdotes.

A note on reporting structure This is a new function at Ro, and we're being deliberate about not over-defining it.
Your manager and where you sit on the org chart will depend on the specific shape of the team we end up with.
We'd rather find the right people and figure out the lines around them than pre-draw boxes and miss great candidates.
These considerations may cause your compensation to vary.

If that ambiguity is a deal-breaker, this isn't the right role; if it sounds like an opportunity, we want to talk. - The target base salary for this position ranges from $182,300 to $220,000, in addition to a competitive equity and benefits

Join Tech @ Ro to build the future of healthcare, from the ground up!
At Ro, we believe that when people achieve their health goals, they can achieve their life goals.
The highest-leverage way to move society forward is to give people their health, and the current healthcare system isn't built to do that.
We've built a vertically integrated healthcare platform that connects telehealth, diagnostics, pharmacy, and logistics into a seamless, end-to-end experience for millions of patients.
Experience with evaluation platforms (e.g. Braintrust, LangSmith, OpenAI Evals), experimentation platforms, causal inference, healthcare, or operations-heavy environments.
Ro is consistently recognized as a top workplace in Health Care, in New York, and for Women and Parents-earning more than 20 honors from Fortune, Great Place to Work, and PEOPLE since 2021.
In 2025 alone, we ranked top 5 among medium workplaces in Health Care and New York, and top 50 nationwide.

This listing is sourced directly from Ro's careers page and normalized into a canonical job model.