Abstract: 

Delivery Hero is one of the leading food delivery platforms that help its customers order food by giving personalized recommendations. The recommendation algorithms used, cater to different use cases: completing the cart, similar product recommendations, complementary product recommendations, etc. The algorithms are powered by advanced data science techniques like gradient boosting, NLP, representation learning to create relevant recommendations. In order to improve user experience we need to constantly improve our algorithms. These improvements could be accurately measured via an online A/B test. Since testing online is always limited by time, the number of users on a platform, and the risk of negatively impacting the user experience which could further impact the business metrics, offline evaluation is used to alleviate this by deciding the best algorithm based on the business objectives.

Key Take Aways:
+ Offline Evaluation
++ Different ways of Offline evaluating recommendation algorithms (depending on the use case and data availability)
++ Usage of traditional evaluation metrics and creating custom metrics based on the recommendation use-case and business objectives
++ Deciding the best recommendation algorithm for the online A/B test

+ Online A/B test
++ Basics of designing an online test
++ Choosing the right statistical test (traditional t-test, bayesian test, bootstrapping, two-part test) based on the type of evaluation metric

By the end of this session, attendees will have a decent understanding of how to conduct offline, online evaluation of recommender systems, enabling them to decide the best algorithm for their use cases.

Background Knowledge:

Python and beginner understanding of recommender systems

Bio: 

Manchit is a Data Science professional with 7 years of experience in the vast field of recommender systems. He has worked in Fashion, and Food domains with industry leaders such as Myntra, and Delivery Hero. Being an applied researcher, he has published papers in esteemed conferences like RecSys, KDD, and ECML-PKDD and has been a reviewer for ECML-PKDD.