ML Inference on Edge with ONNX Runtime
ML Inference on Edge with ONNX Runtime


ONNX Runtime is the inference engine for inferencing ML models in different HW environments. Applications of AI are everywhere. This requires ML models that are trained in the cloud to execute on small devices with low power, low compute, and low memory. Such devices are typically used in IoT scenarios. The data captured by these devices are processed before sending the telemetry to the cloud for further actions on the business application. ONNX Runtime has made enhancements to enable execution of ML models in these edge devices to power AI on the edge applications. This session will walk through the workflow to train an image classification model, package in container and deploy to IoT device.

Session Outline
Lesson 1:
Train ML models for IOT applications, e.g. image classification. Start with a pre-trained model to fine tune for specific IOT scenario. Store model in registry. And convert to ONNX.
Lesson 2:
Create the IoT application in Python using the ML model with ONNX Runtime. Package this in a docker image for the target device and register image in container registry.
Lesson 3:
Deploy the container image to the edge device. Run inference sessions and send processed telemetry to cloud for the business application.

Background Knowledge
- Python;
- Machine Learning life-cycle.


Wolfgang Pauli is an AI developer at Microsoft, with 15+ years of experience with Machine Learning and Artificial Intelligence research. He received his Ph.D. in Computational Neuroscience from the University of Colorado and has published numerous high-profile articles in scientific journals on Computational Neuroscience, Reinforcement Learning, and Neural Networks. Before joining the Microsoft AI Platform team in 2018, he was a research scientist at the California Institute of Technology. He supports the democratization of AI by developing open-source solutions that apply recent breakthroughs to real-world problems.

Open Data Science

Open Data Science
Innovation Center
101 Main St
Cambridge, MA 02142

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Consent to display content from Youtube
Consent to display content from Vimeo
Google Maps
Consent to display content from Google