Abstract

Recent advances in large language models (LLMs) have achieved remarkable results across a wide range of natural language processing (NLP) applications, including text classification, summarization, machine translation, and dialogue systems. As LLMs grow increasingly capable, the need to control their generation process becomes more pressing, particularly for high-stakes applications that demand reliable outputs adhering to specific guidelines or creative outputs within defined boundaries. However, the dominant auto-regressive paradigm—training models to predict the next word based on prior context—poses significant challenges for enforcing structural or content-specific constraints.

In this talk, I will present our recent work on controllable natural language generation (NLG) that moves beyond the conventional auto-regressive framework to enhance both the reliability and creativity of generative models. I will introduce controllable decoding-time algorithms that guide auto-regressive models to better align with user-specified constraints. Additionally, I will discuss a novel insertion-based generation paradigm that breaks away from the limitations of auto-regressive methods. These approaches enable more reliable and creative outputs, with applications spanning creative writing, lexical-controlled generation, and commonsense-compliant text generation.

 Bio

Nanyun (Violet) Peng is an Associate Professor of Computer Science at The University of California, Los Angeles. She received her Ph.D. from the Center for Language and Speech Processing at Johns Hopkins University. Her research focuses on controllable and creative language generation, multilingual and multimodal models, and the development of automatic evaluation metrics, with a strong commitment to advancing robust and trustworthy artificial intelligence (AI). Her work has been recognized with honors such as an Outstanding Paper Award at NAACL 2022, three Outstanding Paper Awards at EMNLP 2024, Oral Paper selections at NeurIPS 2022 and ICML 2023, as well as several Best Paper Awards at workshops affiliated with premier AI and NLP conferences. She was also featured in the IJCAI 2022 Early Career Spotlight. Her research has received support from prestigious funding sources, including an NSF CAREER Award, NIH R01, DARPA, IARPA grants, and multiple industrial research awards.