Abstract

Large Language Models (LLMs) have become ubiquitous across various domains, transforming the way we interact with information and conduct research. While the proliferation of LLMs has enhanced numerous applications, a significant number of high-performing models remain proprietary, impeding the progress of scientific exploration. LLMs are also susceptible to hallucinations, generating seemingly credible yet factually inaccurate information that can impact their broad acceptance and integration. In this seminar, I will commence by introducing one of our open-sourced XGen LLMs. I will delve into its pre-training process and present its results on standard benchmarks. Subsequently, I will discuss our work involving reasoning with LLMs, democratizing them for low-resource languages, and distilling knowledge from a larger (175B) proprietary LLM to a smaller (7B) model in a personalized manner. Finally, I will conclude by addressing some limitations of LLMs, emphasizing that scaling alone might not suffice as a solution and that new innovations are needed to tackle these challenges.

Bio