What to expect when you’re putting AI in production
What to expect when you’re putting AI in production


Machine learning and “AI” systems often fail in production in unexpected ways. This talk shares real-world case studies showing why this happens and explains what you can do about it, covering best practices and lessons learned from a decade of experience building and operating such systems at Fortune 500 companies across several industries.

The covered topics include concept drift (identifying and correcting for model decay due to changes in the distribution of data in production), common pitfalls in A/B testing (like the primacy and novelty effects), offline versus online measurements, and systems that learn in production (such as adversarial learning use cases). This talk is intended for executives, technical leaders and product managers who want to learn from others’ mistakes how to best set up their teams & products for success.


David Talby has been building real-world big data analytics systems in healthcare, finance and e-commerce for over a decade. David has extensive experience in building and operating web-scale data science and business platforms, as well as building world-class, Agile, distributed teams. Prior to joining the startup world, he was with Microsoft’s Bing group, where he led business operations for Bing Shopping in the US and Europe. Earlier, he worked at Amazon both in Seattle and the UK, where he built and ran distributed teams that helped scale Amazon’s financial systems. David holds a PhD in computer science and master’s degrees in both computer science and business administration.

Open Data Science




Open Data Science
One Broadway
Cambridge, MA 02142

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Consent to display content from - Youtube
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google