Error Analysis for Accelerating Responsible Machine Learning
Error Analysis for Accelerating Responsible Machine Learning

Abstract: 

As Artificial Intelligence is becoming part of user-facing applications and directly impacting society, deploying AI reliably and responsibly has become a priority for Microsoft and several other industry leaders. Rigorous model evaluation and debugging are at the heart of responsible machine learning development. Yet, many of the standard practices continue to focus on high-level aggregated evaluation that uses only single accuracy numbers to report model accuracy on large benchmarks. Such practices can be misleading because they hide important failure modes that happen either for unexpected input conditions, corner cases, or specific demographic groups. At the same time, understanding where hidden pockets of errors lie in large data manifolds can be tedious and time consuming for practitioners. This presentation will introduce Error Analysis to the audience, as a tool and methodology for effectively identifying and diagnosing errors in machine learning models, beyond aggregated accuracy scores. The tool provides different views for quick error identification and enables error diagnosis either via active data exploration or model explanations generated using the InterpretML library. We will deep dive into the tool functionalities by presenting case studies and a live step-by-step demo. Finally, we will conclude with a discussion on future opportunities we are considering on further integrations with other RAI tools, as a quest towards a better integrated RAI ecosystem.

Website: erroranalysis.ai
Github repository: github.com/microsoft/responsible-ai-widgets
Practitioner-oriented blog on error analysis: https://techcommunity.microsoft.com/t5/azure-ai/responsible-machine-learning-with-error-analysis/ba-p/2141774

Bio: 

Mehrnoosh Sameki is a senior technical program manager at Microsoft, responsible for leading the product efforts on machine learning interpretability and fairness within the Azure Machine Learning platform. She earned her PhD degree in computer science at Boston University, where she currently serves as an adjunct assistant professor and lecturer, offering courses in responsible AI. Previously, she was a data scientist in the retail space, incorporating data science and machine learning to enhance customers’ personalized shopping experiences.

Open Data Science

Open Data Science
Innovation Center
101 Main St
Cambridge, MA 02142
info@odsc.com

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Youtube
Consent to display content from Youtube
Vimeo
Consent to display content from Vimeo
Google Maps
Consent to display content from Google