PyTorch 101: building a model step-by-step


Learn the basics of building a PyTorch model using a structured, incremental and from first principles approach. Find out why PyTorch is the fastest growing Deep Learning framework and how to make use of its capabilities: autograd, dynamic computation graph, model classes, data loaders and more.

The main goal of this session is to show you how PyTorch works: we will start with a simple and familiar example in Numpy and "torch" it! At the end of it, you should be able to understand PyTorch's key components and how to assemble them together into a working model.

Session Outline:
Module 1: PyTorch: tensors, tensors, tensors
• Introducing a simple and familiar example: linear regression
• Generating synthetic data
• Tensors: what they are and how to create them
• CUDA: GPU vs CPU tensors
• Parameters: tensors meet gradients

Module 2: Gradient Descent in Five Easy Steps
• Step 0: initializing parameters
• Step 1: making predictions in the forward pass
• Step 2: computing the loss, or “how bad is my model?”
• Step 3: computing gradients, or “how to minimize the loss?”
• Step 4: updating parameters
• Bonus: learning rate, the most important hyper-parameter
• Step 5: Rinse and repeat

Module 3: Autograd, your companion for all your gradient needs! (15 min)
• Computing gradients automatically with the backward method
• Dynamic Computation Graph: what is that?
• Optimizers: updating parameters, the PyTorch way
• Loss functions in PyTorch

Module 4: Building a Model in PyTorch
• Your first custom model in PyTorch
• Peeking inside a model with state dictionaries
• The importance of setting a model to training mode
• Nested models, layers, and sequential models
• Organizing our code: the training step

Module 5: Datasets and data loaders
• Your first custom dataset in PyTorch
• Data loaders and mini-batches
• Evaluation phase: setting up the stage
• Organizing our code: the training loop
• Putting it all together: data preparation, model configuration, and model training
• Taking a break: saving and loading models

Background Knowledge:
We will use Google Colab and work our way together into building a complete model in PyTorch. You should be comfortable using Jupyter notebooks, Numpy and, preferably, object oriented programming.


Daniel has been teaching machine learning and distributed computing technologies at Data Science Retreat, the longest-running Berlin-based bootcamp, for more than three years, helping more than 150 students advance their careers. He writes regularly for Towards Data Science. His blog post "Understanding PyTorch with an example: a step-by-step tutorial" reached more than 220,000 views since it was published. The positive feedback from the readers motivated him to write the book Deep Learning with PyTorch Step-by-Step, which covers a broader range of topics. Daniel is also the main contributor of two python packages: HandySpark and DeepReplay. His professional background includes 20 years of experience working for companies in several industries: banking, government, fintech, retail and mobility.

Open Data Science




Open Data Science
One Broadway
Cambridge, MA 02142

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Consent to display content from Youtube
Consent to display content from Vimeo
Google Maps
Consent to display content from Google