Experimental Reproducibility in Data Science with Sacred

Abstract: There are ways to incorporate experimental reproducibility into machine learning projects that are clean and lightweight. In this introductory level workshop, we demonstrate how to use Sacred to motivate reproducible research and experiment monitoring in machine learning. We discuss how this enables any data scientist to provide a solution (a model or set of predictions) to any problem, compare their solution to previous models results on the same test data, and select the best model for production. Finally, we provide examples of machine learning problems in retail and demonstrate how data scientists can easily work across multiple problems.

Bio: Karthik is a Data Scientist at Gilt working on predicting customer demand and lifetime value. Karthik has a background in mathematics, and worked a few years abroad in Shanghai working on machine learning problems in edtech and fintech.

Open Data Science Conference