Testing Production Machine Learning Systems


Testing is a critical part of the software development cycle. As your software project grows, dealing with bugs and regressions can consume your team if you do not take a principled approach to testing. As a result, software testing methodologies are well-studied. However, machine learning models introduce a new set of complexities beyond traditional software. In particular, machine learning models depend on data in addition to code. As a result, testing methodologies for machine learning systems are less understood and less widely implemented in practice. In this talk, we argue for the importance of testing in ML, give an overview of the types of testing available to ML practitioners, and make recommendations about how you can start to incorporate more robust testing into your ML projects.


Josh is the Founder of Stealth Startup and his research focuses on applying deep reinforcement learning, generative models, and synthetic data to problems in robotic perception and control. Additionally, he co-organizes a machine learning training program for engineers to learn about production-ready deep learning called Full Stack Deep Learning. Previously, Josh was a Research Scientist at OpenAI working at the intersection of machine learning and robotics. Josh did his PhD in Computer Science at UC Berkeley advised by Pieter Abbeel. He have also been a management consultant at McKinsey and an Investment Partner at Dorm Room Fund.

Open Data Science




Open Data Science
One Broadway
Cambridge, MA 02142

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Consent to display content from - Youtube
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google