Ml Inference on Mobile Device With Onnx Runtime
Ml Inference on Mobile Device With Onnx Runtime

Abstract: 

AI model inference on the phone is important to deliver real-time experience that requires execution locally on the device. But there are many constraints to build a scalable solution addressing all the different hardware specs and platform requirements across Android and iOS. Devices have limited storage and memory, and the platform app stores impose restrictions on the size of the app package.
ONNX Runtime Mobile is a new feature to address the needs for developers to build solutions for Mobile devices. You can build a reduced size binary package to integrate with the phone application and inference your ONNX models locally on devices.
ONNX Quantization techniques can be used to reduce the model size by converting FP32 weights to INT8. Improved performance is achieved with INT8 execution on ARM processors on the mobile phone.

Bio: 

Manash Goswami is a product management leader with a passion to solve problems and build products that meet end-user needs. Grown from development - coding, test & verification, systems design & architecture, to product management and business development. Delivered consumer electronics products from smartwatch to tablets. Deep understanding of the Android, Chrome & Windows eco-system. Demonstrated leadership by managing through influence across a matrixed org structure. Capable of driving new initiative/agenda internally or with customers, enable collaboration across functional teams and drive for financial results. Excel at dealing with ambiguity and managing complex, cross-group interactions. Can successfully tie long-term strategic thinking with near-term execution.

Open Data Science

Open Data Science
Innovation Center
101 Main St
Cambridge, MA 02142
info@odsc.com

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Youtube
Consent to display content from Youtube
Vimeo
Consent to display content from Vimeo
Google Maps
Consent to display content from Google