Abstract: 

The real world is open and full of unknowns, presenting significant challenges for AI systems that must reliably handle diverse, and sometimes anomalous inputs. Out-of-distribution (OOD) uncertainty arises when a machine learning model sees a test-time input that differs from its training data, and thus should not be predicted by the model. As ML is used for more safety-critical domains, the abilities to handle out-of-distribution data are central in building open-world learning systems. In this talk, I will talk about methods, challenges and opportunities towards building ROWL (Reliable Open-World Learning).

To tackle these challenges, I will first describe mechanism that improves OOD uncertainty estimation by using calibrated softmax score and input processing. I will then talk about recent advancement of an energy-based OOD detection framework, which produces theoretically grounded measurement that is aligned with the probability density of the input data. We show that energy score is less susceptible to softmax's overconfidence issue, and leads to superior performance on common OOD detection benchmarks. Lastly, I will discuss how to scale out-of-distribution detection algorithms to real-world large-scale classification problems.

Bio: 

Sharon Yixuan Li is an Assistant Professor in the Department of Computer Sciences at University of Wisconsin Madison. Previously she spent a wonderful year as a postdoc researcher in the Computer Science department at Stanford University, working with Chris Ré. Sharon Yixuan Li completed her PhD from Cornell University in 2017, where Sharon was fortunate to be advised by John E. Hopcroft. Her thesis committee members are Kilian Q. Weinberger and Thorsten Joachims. Sharon has spent time at Google AI twice as an intern, and Facebook AI as a Research Scientist. Sharon was named Forbes 30 Under 30 in Science in 2020.

Sharon Yixuan Li broad research interests are in deep learning and machine learning. Her time in both academia and industry has shaped her view and approach in research. The goal of the research is to enable transformative algorithms and practices towards reliable open-world learning, which can function safely and adaptively in the presence of evolving and unpredictable data stream. Our works explore, understand, and mitigate the many challenges where failure modes can naturally occur in deploying machine learning models in the open world.