AI Evaluation Scientist
Location
McLean, VA
Job Type
Full-time
Category
other-general
Posted
June 17, 2026
**Overview**
We are looking for an **AI Evaluation Scientist** to design and execute evaluation processes that ensure our predictive and generative AI systems are accurate, reliable, safe, and aligned with mission requirements. This role is essential for establishing trust in AI solutions and supporting continuous improvement across the AI lifecycle. The AI Evaluation Scientist will work closely with engineers, data scientists, governance analysts, and product teams to develop evaluation metrics, build test harnesses, analyze model behavior, and support responsible deployment.
**Contributions**
+ Implement evaluation frameworks for AI models, including accuracy, robustness, relevance, bias, hallucination rate, and safety metrics.
+ Build and maintain automated evaluation scripts, tests, and pipelines that assess AI model outputs and detect performance drift over time.
+ Develop benchmark datasets, challenge sets, and scenario-based test cases tailored...
We are looking for an **AI Evaluation Scientist** to design and execute evaluation processes that ensure our predictive and generative AI systems are accurate, reliable, safe, and aligned with mission requirements. This role is essential for establishing trust in AI solutions and supporting continuous improvement across the AI lifecycle. The AI Evaluation Scientist will work closely with engineers, data scientists, governance analysts, and product teams to develop evaluation metrics, build test harnesses, analyze model behavior, and support responsible deployment.
**Contributions**
+ Implement evaluation frameworks for AI models, including accuracy, robustness, relevance, bias, hallucination rate, and safety metrics.
+ Build and maintain automated evaluation scripts, tests, and pipelines that assess AI model outputs and detect performance drift over time.
+ Develop benchmark datasets, challenge sets, and scenario-based test cases tailored...