Applied AI Evaluation Engineer

About the Role

As an Engineering & Data Science Specialist in Applied AI Evaluations, you will design, implement, and optimize evaluation frameworks and pipelines that measure the quality, safety, and performance of advanced AI systems at BMO. You’ll work closely with the Product Owner and collaborate with engineering, research, and business teams to translate evaluation standards into robust, scalable solutions.

Your work will directly impact model development, deployment, and continuous improvement cycles—ensuring that evaluation signals are deeply integrated into experimental and production systems. You’ll contribute to the development of reusable tools, agents, and harnesses that support responsible AI practices and regulatory compliance.

Key Responsibilities