Building a ML Serving Platform at Scale for Natural Language Processing


NLP driven AI is a fast-growing technology domain with diverse applications in customer engagement, employee collaboration, marketing, and social media. Does having an accurate model by itself mean, that we have a successful product, service, or solution? No! The most difficult phase begins after that. We still have the following challenges to solve. How do we Create a sellable service around this model? How to Scale this service to handle millions of inference requests, in a cost-effective manner? How to build automated deployment pipelines for software and models? How to add security, privacy, manageability, and observability for the service? How to track model drift and analyze model performance? In this session, we will discuss these questions and explore answers. We will go through the unique challenges that NLP serving poses and the solutions and best practices to overcome them.


Kumaran Ponnambalam is an AI and Big Data leader with 15+ years of experience. He is currently the Director of AI for Webex Contact Center at Cisco. He focuses on creating robust, scalable AI platforms and models to drive effective customer engagements. In his current and previous roles, he has built data pipelines, ML models, analytics and integrations around customer engagement. He has also authored several courses on the LinkedIn Learning Platform in Machine Learning and Big Data areas. He holds a MS in Information Technology and advanced certificates in Deep Learning and Data Science.

Open Data Science




Open Data Science
One Broadway
Cambridge, MA 02142

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Consent to display content from - Youtube
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google