Framework for Model Monitoring at Scale

Abstract: 

Being able to effectively monitor production models in real-time and respond to signals is critical to the success of any organizational data science investment. Yet, this seems to be a mountain too many organizations can’t climb. Why?

The reality is: It’s one thing to build a system that technically checks the monitoring box. It’s quite another to consider all the complexities of maintaining a monitoring program, especially at scale. How can we assert that every model is monitored? Who should own the monitoring system? What metrics are best for early signal detection? The barriers to success are many and varied. They are both technical and process-related.

In this talk, James Pearce, a long-time data science leader, will share first-hand perspectives on the importance of having a model monitoring framework in place, and lessons he’s learned while building out this capability at a large national bank. Josh Poduska, Chief Data Scientist at Domino Data Lab, will demonstrate one approach to incorporating the technical and process-related best practices of model monitoring to help increase your odds of successfully implementing a monitoring program.

Bio: 

Josh Poduska is the Chief Data Scientist at Domino Data Lab. He has 18 years of experience in the analytics space, particularly with designing and implementing data science solutions in the manufacturing and public sector domains. He has also led data science teams and strategy for several analytical software companies. Josh has a Masters in Applied Statistics from Cornell University.

Open Data Science

Open Data Science
Innovation Center
101 Main St
Cambridge, MA 02142
info@odsc.com

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Youtube
Consent to display content from Youtube
Vimeo
Consent to display content from Vimeo
Google Maps
Consent to display content from Google