Supporting Your Machine Learning Teams: Testing, Modularity and Monitoring


As machine learning in production becomes the norm, it is critical we help our teams prepare for supporting and iterating on our models. While we have much to learn from the best practices of shipping software, we must also recognize the nuances required to support AI.

What differentiates AI from traditional software engineering? The conversation often focuses on the data, the model quality, and the massive business impact it will have once deployed. But moving beyond the individual models and their first deployment exists the same realities as with all software: the need for iteration, quality and scale. Here, we must blend the expertise of the engineers with the nuanced needs of our data scientists.

In this talk, we will dive into the specifics, choosing targets within the ML-development lifecycle where we have much to learn from each other. How can testing frameworks be used, and further adapted, to help us more safely iterate on our models in production? What role does modularity play in helping us be more agile? What does it mean to monitor models once they are deployed?

For each of these questions we will explore the opportunities at hand for leverage from existing best practices, but discuss how they must be adapted to the realities of AI.


Sarah Aerni is a Senior Manager of Data Science at Salesforce Einstein, where she leads teams building AI-powered applications across the Salesforce platform. Prior to Salesforce she led the healthcare & life science and Federal teams at Pivotal. Sarah obtained her PhD from Stanford University in Biomedical Informatics, performing research at the interface of biomedicine and machine learning. She also co-founded a company offering expert services in informatics to both academia and industry.

Open Data Science




Open Data Science
One Broadway
Cambridge, MA 02142

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Consent to display content from - Youtube
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google