How to Systematically Evaluate and Improve your Generative AI Application

Abstract: 

In this demo-centric session, we'll start by showing you a generative AI app that brings our own data to an LLM, using Retrieval-Augmented Generation (RAG). We’ll then show you a systematic approach for developing, measuring, and improving generative AI applications, using Microsoft's PromptFlow. We'll evaluate the quality of our current application according to different metrics, we'll make changes to our logic accordingly, and we'll re-evaluate our changes to quantify the improvements made.

Bio: 

Daniel Schneider is Data Scientist at Microsoft in the AzureML team. He has a background in computer science and machine learning and has worked on different products at Microsoft, including Skype, Windows, and Azure.

Open Data Science

 

 

 

Open Data Science
One Broadway
Cambridge, MA 02142
info@odsc.com

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Youtube
Consent to display content from - Youtube
Vimeo
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google