Personalizing LLMs with a Feature Store

Abstract: 

Large language models (LLMs) encode a model language, and by extension the world. Prompt engineering can be used to personalize LLMs by pre-pending a prompt to the user's LLM query. Feature stores are already widely used to provide context and history to online models that are deployed on stateless applications - an application can use a user-id or session-id to retrieve precomputed features for the user or session that are used to enrich the features available at request-time. We can easily imagine how the answer provided by a LLM to a task about how get to my summer house could be improved if the LLM knew the time of year, the weather, that I like to cycle that distance, and that I have a car.

In this talk, we show how to personalize LLMs using a feature store and prompt engineering. We walk through how to build an example free serverless, personalized LLM application using Hopsworks, an open-source feature store with a built-in vector database. We will look at how to build templates for prompts, and how they can be easily constructed and included in user queries. We will look at how to fill-in prompt templates with real-time context data, produced by streaming feature pipelines, and user-specific data, produced by batch feature pipelines. We will also look at how we can incorporate documents from vector databases in prompts using a combination of user-input and historical user data from the feature store. We will make this example application available as open-source in Python.

Bio: 

Fabio Buso is a co-founder and VP of Engineering at Hopsworks, leading the Feature Store development team. Fabio holds a master’s degree in Cloud Computing and Services with a focus on data intensive applications.

Open Data Science

 

 

 

Open Data Science
One Broadway
Cambridge, MA 02142
info@odsc.com

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Youtube
Consent to display content from - Youtube
Vimeo
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google