Integrating Language Models for Automating Feature Engineering Ideation


Feature engineering has long relied on manual expertise, demanding domain knowledge and experience. While automated approaches exist, they often involve brute-force methods and subsequent feature selection. The emergence of Large Language Models (LLMs) offers a novel perspective on feature engineering. LLMs encapsulate extensive knowledge, presenting an opportunity to reshape this process.

In this presentation, we explore an approach that utilizes LLMs to guide feature engineering. By leveraging the contextual understanding within LLMs, we have developed a system for LLM-assisted feature engineering. Our research demonstrates the practical benefits of this synergy – from feature ideation to improved feature relevance to enhanced model interpretability and efficiency.

Join us to discover how our framework automates and enhances feature engineering, contributing to better predictive models. This talk showcases the integration of human expertise with language models, revolutionizing feature engineering in data science.


Sergey is a data scientist with a background in physics and neurobiology. FeatureByte is Sergey's second startup. He was one of the first employees at DataRobot where he created and led a professional services group and helped the company grow into a unicorn. Sergey is widely known for being a Kaggle Grandmaster and holding the #1 rank on Kaggle in the past. Multiple times he was mentioned as one of the top data scientists by various publications. Sergey’s passion is in machine learning, predictive modeling and inventive feature engineering.

Open Data Science




Open Data Science
One Broadway
Cambridge, MA 02142

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Consent to display content from - Youtube
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google