Abstract: Learn the basics of building a PyTorch model using a structured, incremental and from first principles approach. Find out why PyTorch is the fastest growing Deep Learning framework and how to make use of its capabilities: autograd, dynamic computation graph, model classes, data loaders and more.
The main goal of this session is to show you how PyTorch works: we will start with a simple and familiar example in Numpy and "torch" it! At the end of it, you should be able to understand PyTorch's key components and how to assemble them together into a working model.
Module 1: PyTorch: tensors, tensors, tensors
Introducing a simple and familiar example: linear regression
Generating synthetic data
Tensors: what they are and how to create them
CUDA: GPU vs CPU tensors
Parameters: tensors meet gradients
Module 2: Gradient Descent in Five Easy Steps
Step 0: initializing parameters
Step 1: making predictions in the forward pass
Step 2: computing the loss, or how bad is my model?
Step 3: computing gradients, or how to minimize the loss?
Step 4: updating parameters
Bonus: learning rate, the most important hyper-parameter
Step 5: Rinse and repeat
Module 3: Autograd, your companion for all your gradient needs! (15 min)
Computing gradients automatically with the backward method
Dynamic Computation Graph: what is that?
Optimizers: updating parameters, the PyTorch way
Loss functions in PyTorch
Module 4: Building a Model in PyTorch
Your first custom model in PyTorch
Peeking inside a model with state dictionaries
The importance of setting a model to training mode
Nested models, layers, and sequential models
Organizing our code: the training step
Module 5: Datasets and data loaders
Your first custom dataset in PyTorch
Data loaders and mini-batches
Evaluation phase: setting up the stage
Organizing our code: the training loop
Putting it all together: data preparation, model configuration, and model training
Taking a break: saving and loading models
We will use Google Colab and work our way together into building a complete model in PyTorch. You should be comfortable using Jupyter notebooks, Numpy and, preferably, object oriented programming.
Bio: Daniel has been teaching machine learning and distributed computing technologies at Data Science Retreat, the longest-running Berlin-based bootcamp, for more than three years, helping more than 150 students advance their careers. He writes regularly for Towards Data Science. His blog post "Understanding PyTorch with an example: a step-by-step tutorial" reached more than 220,000 views since it was published. The positive feedback from the readers motivated him to write the book Deep Learning with PyTorch Step-by-Step, which covers a broader range of topics. Daniel is also the main contributor of two python packages: HandySpark and DeepReplay. His professional background includes 20 years of experience working for companies in several industries: banking, government, fintech, retail and mobility.