Reliable Open-World Learning Against Out-of-distribution Data
Reliable Open-World Learning Against Out-of-distribution Data

Abstract: 

The real world is open and full of unknowns, presenting significant challenges for AI systems that must reliably handle diverse, and sometimes anomalous inputs. Out-of-distribution (OOD) uncertainty arises when a machine learning model sees a test-time input that differs from its training data, and thus should not be predicted by the model. As ML is used for more safety-critical domains, the abilities to handle out-of-distribution data are central in building open-world learning systems. In this talk, I will talk about methods, challenges and opportunities towards building ROWL (Reliable Open-World Learning).

To tackle these challenges, I will first describe mechanism that improves OOD uncertainty estimation by using calibrated softmax score and input processing. I will then talk about recent advancement of an energy-based OOD detection framework, which produces theoretically grounded measurement that is aligned with the probability density of the input data. We show that energy score is less susceptible to softmax's overconfidence issue, and leads to superior performance on common OOD detection benchmarks. Lastly, I will discuss how to scale out-of-distribution detection algorithms to real-world large-scale classification problems.

Bio: 

Sharon Yixuan Li is an Assistant Professor in the Department of Computer Sciences at University of Wisconsin Madison. Previously she spent a wonderful year as a postdoc researcher in the Computer Science department at Stanford University, working with Chris Ré. Sharon Yixuan Li completed her PhD from Cornell University in 2017, where Sharon was fortunate to be advised by John E. Hopcroft. Her thesis committee members are Kilian Q. Weinberger and Thorsten Joachims. Sharon has spent time at Google AI twice as an intern, and Facebook AI as a Research Scientist. Sharon was named Forbes 30 Under 30 in Science in 2020.

Sharon Yixuan Li broad research interests are in deep learning and machine learning. Her time in both academia and industry has shaped her view and approach in research. The goal of the research is to enable transformative algorithms and practices towards reliable open-world learning, which can function safely and adaptively in the presence of evolving and unpredictable data stream. Our works explore, understand, and mitigate the many challenges where failure modes can naturally occur in deploying machine learning models in the open world.

Open Data Science

Open Data Science
Innovation Center
101 Main St
Cambridge, MA 02142
info@odsc.com

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Youtube
Consent to display content from Youtube
Vimeo
Consent to display content from Vimeo
Google Maps
Consent to display content from Google