Abstract: 

PyTorch is a very popular deep learning framework underpinning recent AI advancements. While its origins are in research, PyTorch is used for production applications across many domains. One popular application category is ranking and recommendation systems that are ubiquitous in online products we interact with every day.

By completing this workshop, you will refresh the basics of neural network authoring in PyTorch and learn tools for deploying the models to production. You will learn some of the most popular deep learning models architectures for recommender systems and how to use TorchRec to train them efficiently.

Session Outline:

Module 1. Refresher of training neural networks in PyTorch. You will exercise building a simple neutral network training from scratch: model architecture, data loader, optimization loop, etc for a simple task with tabular data for now.

Module 2. Modeling for deep recommender systems. Learn major architecture types: combining dense and sparse features, two-tower architectures for efficient retrieval, and sequence modeling approaches. Implement the simple DLRM architecture from scratch using TorchRec library.

Module 3. Scale and production. Learn PyTorch Profiler to understand the performance of PyTorch models and get started with distributed training. Apply optimized building blocks from TorchRec to speed up training and scale up model size.

Background Knowledge:

Very basic familiarity with deep learning in Python is assumed (like walking through a toy model tutorial in PyTorch/Tensorflow/Keras/etc).

Bio: 

Dmytro Dzhulgakov is a technical lead and a core maintainer of PyTorch. He’s been focusing on the framework core development and production deployment for the past 5 years. Previously, he co-created the ONNX interoperable AI initiative. Dmytro built several generations of large-scale deep-learning recommendation systems at Meta powering Ads and News Feed.