Abstract: 

Large language models (LLMs) have achieved a milestone that undeniably changed many held beliefs in artificial intelligence (AI). However, there re-mains many limitations of these LLMs when it comes to true language un-derstanding, limitations that are a byproduct of the underlying architecture of deep neural networks. Moreover, and due to their subsymbolic nature, whatever knowledge these models acquire about how language works will always be buried in billions of microfeatures (weights), none of which is meaningful on its own, making such models hopelessly unexplainable. To address these limitations, we suggest combining the strength of symbolic representations with what we believe to be the key to the success of LLMs, namely a successful bottom-up reverse engineering of language at scale. As such we argue for a bottom-up reverse engineering of language in a symbol-ic setting. Hints on what this project amounts to have been suggested by several authors, and we discuss in some detail here how this project could be accomplished.

Learning objectives: Why we need Symbolic and Explainable Large Language Models (LLMs)

Bio: 

Walid Saba is a Senior Research Scientist at the Institute for Experiential AI at Northeastern University. Prior to joining the institute in 2023, he worked at two Silicon Valley startups, focusing on conversational AI. This work included high-level roles as the principal AI scientist for telecommunications company Astound and CTO of software company Klangoo, where he helped develop its state-of-the-art digital content semantic engine (Magnet).

Saba’s career to date has seen him hold various positions in both the private sector and academia. His resume includes entities such as the American Institutes for Research, AT&T Bell Labs, IBM and Cognos, while he has also spent a cumulative seven years teaching computer science at the University of Ottawa, the New Jersey Institute of Technology (NJIT), the University of Windsor (a public research university in Ontario, Canada), and the American University of Beirut (AUB).

Walid is frequent invited for interviews and as a keynote speaker on AI and NLP has published over 45 technical articles, including an award-winning paper that he presented at the German Artificial Intelligence Conference (KI-2008). Walid received his BSc and MSc in Computer Science from the University of Windsor, and a Ph.D in Computer Science from Carleton University in 1999.