Abstract: 

Only 15% of AI projects will yield results in 2022. That's bad. The good news: there is a better way. We can deliver high-level quality AI systems that meet business objectives and drive adoption.

In this tutorial, Olivier Blais, Project Editor of the ""ISO/IEC TS 5471 – Quality evaluation guidelines for AI Systems"" technical specifications, will share modern best practices and techniques to improve the quality evaluation of AI systems. This session targets AI practitioners, AI project managers, and AI leads as AI system quality is paramount in every AI project, delivery processes and experts’ toolbox. Participants will learn about upcoming AI system quality evaluation approach and methods that will inspire future certifications.

Session Outline

Lesson 1: Introduction to quality evaluation of an AI system
General overview of what does mean to have an AI system of good quality. On top of that, the AI systems and software quality requirements and evaluation (SQuaRE for AI systems) framework will be introduced and explained.

Lesson 2: Summary of key components of a modern AI system quality evaluation approach
Presentation of critical characteristics of AI system quality and their most essential measurement methods. More specifically, the following characteristics will be described in more detail:

Correctness
Adaptability
Robustness
Completeness
Robustness
Lesson 3: A live demo of the AI system quality evaluation framework
Let’s now apply the quality evaluation framework and the main measurement methods to an actual machine learning model.

Background Knowledge

It is assumed that the audience has a some understanding or experience about the development of an AI project. However, some advanced concepts and measurement methods will be presented.

Bio: 

Olivier is co-founder and VP of decision science at Moov AI. He is the editor of the international ISO standard that defines the quality of artificial intelligence systems, where he leads a team of 50 AI professionals from around the world.

His cutting-edge AI and machine learning knowledge have led him to implement a data culture in various industries and support digital transformation projects in many companies such as Pratt & Whitney, Metro, Sharethrough, Merck, and Premier Tech.

He is a mentor for AI for Creative Destruction Labs and coaches several start-ups. As a speaker, his topics of choice are adopting and applying AI and responsible AI.

Olivier is the recipient of the prestigious ""30 under 30"" award (2019) and is co-author of a patent for an advanced algorithm that evaluates a borrower's creditworthiness.