Home
About
People
Publications
Seminars
- Seminar Calendar
- Seminar Videos
Workshops
HLT Masters
Resources

Probabilistic Models of Relational Domains

Add to Calendar

Add to Timely Calendar
Add to Google
Add to Outlook
Add to Apple Calendar
Add to other calendar
Export to XML

When:

August 26, 2003 all-day

2003-08-26T00:00:00-04:00

2003-08-27T00:00:00-04:00

Seminars

View Seminar Video

This post was replicated from another site's calendar feed. View original

Seminar Videos

Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements (Jack Zhang)

Jingyu Jack Zhang: https://jackz.io/

Paper: https://arxiv.org/abs/2410.08968

Abstract: The current paradigm for safety alignment of large language models (LLMs) follows a one-size-fits-all approach: the model refuses to interact with any content deemed unsafe by the model provider. This approach lacks flexibility in the face of varying social norms across cultures and regions. In addition, users may have diverse safety needs, making a model with static safety standards too restrictive to be useful, as well as too costly to be re-aligned. We propose Controllable Safety Alignment (CoSA), a framework designed to adapt models to diverse safety requirements without re-training. Instead of aligning a fixed model, we align models to follow safety configs -- free-form natural language descriptions of the desired safety behaviors -- that are provided as part of the system prompt. To adjust model safety behavior, authorized users only need to modify such safety configs at inference time. To enable that, we propose CoSAlign, a data-centric method for aligning LLMs to easily adapt to diverse safety configs. Furthermore, we devise a novel controllability evaluation protocol that considers both helpfulness and configured safety, summarizing them into CoSA-Score, and construct CoSApien, a human-authored benchmark that consists of real-world LLM use cases with diverse safety requirements and corresponding evaluation prompts. We show that CoSAlign leads to substantial gains of controllability over strong baselines including in-context alignment. Our framework encourages better representation and adaptation to pluralistic human values in LLMs, and thereby increasing their practicality.

Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements (Jack Zhang)

Abstract
Language model pretraining has been a remarkably strong recipe for cross-task and cross-domain generalization in NLP. However, these gains have come at the expense of control: we rarely control the training data for language models, and gaps between pretraining and our target evaluation lead to distribution shifts. We present two complementary approaches to control this gap — algorithmically filtering data to focus training on the most benchmark-relevant parts of the distribution, as well as adapting to new domains by synthesizing domain-specific pretraining data at scale.

Biography
Tatsunori Hashimoto is an Assistant Professor in the Computer Science Department at Stanford University. He is a member of the statistical machine learning and natural language processing groups at Stanford and studies statistical approaches to improving and understanding language models. Work from his group spans many areas, including instruction-following and controllable language models, differentially private fine-tuning, and benchmarks for LM safety and capabilities. He received his Ph.D. at MIT under the supervision of Tommi Jaakkola and David Gifford, and is the recipient of a Kavli fellowship, the NSF CAREER, faculty research awards from Google, Sony and Amazon, and best paper awards at ICML, ICLR, and CHI.

Improving LLM generalization by selecting and synthesizing data -- Tatsu Hashimoto (Stanford)

Paper: https://arxiv.org/abs/2409.08425

Abstract

SoloAudio is an innovative diffusion-based generative model designed for target sound extraction (TSE). It trains latent diffusion models on audio, replacing the previous U-Net backbone with a skip-connected Transformer that operates on latent features. SoloAudio enables both audio-oriented and language-oriented TSE by employing the CLAP model as a feature extractor for target sounds. Additionally, it leverages synthetic audio generated by state-of-the-art text-to-audio models for training, showcasing strong generalization to out-of-domain data and novel sound events. SoloAudio demonstrates impressive zero-shot and few-shot learning capabilities.

Solo Audio: Target Sound Extraction with Language-oriented Audio Diffusion Transformer -- Helin Wang

Abstract
Numerous capability and safety techniques of Large Language Models (LLMs), including RLHF, automated red-teaming, prompt engineering, and infilling, can be cast as sampling from an unnormalized target distribution defined by a given reward or potential function over the full sequence. In this work, we leverage the rich toolkit of Sequential Monte Carlo (SMC) for these probabilistic inference problems. In particular, we use learned twist functions to estimate the expected future value of the potential at each timestep, which enables us to focus inference-time computation on promising partial sequences. We propose a novel contrastive method for learning the twist functions, and establish connections with the rich literature of soft reinforcement learning. As a complementary application of our twisted SMC framework, we present methods for evaluating the accuracy of language model inference techniques using novel bidirectional SMC bounds on the log partition function. These bounds can be used to estimate the KL divergence between the inference and target distributions in both directions. We apply our inference evaluation techniques to show that twisted SMC is effective for sampling undesirable outputs from a pretrained model (a useful component of harmlessness training and automated red-teaming), generating reviews with varied sentiment, and performing infilling tasks.

Biography
Roger Grosse is an Associate Professor of Computer Science at the University of Toronto, Schwartz-Reisman Chair in Technology and Society, founding member of the Vector Institute for Artificial Intelligence, and a Member of Technical Staff on the Alignment Science team at Anthropic. His research focuses on using our understanding of deep learning to improve the safety and alignment of AI systems. He has held the Sloan Research Fellowship, Canada CIFAR AI Chair, and Canada Research Chair.

Probabilistic Inference in Language Models via Twisted Sequential Monte Carlo -- Roger Grosse

Project page: https://generative-world-explorer.github.io/
Paper: https://arxiv.org/abs/2411.11844

Abstract: Planning with partial observation is a central challenge in embodied AI. A majority of prior works have tackled this challenge by developing agents that physically explore their environment to update their beliefs about the world state. In contrast, humans can imagine unseen parts of the world through a mental exploration and revise their beliefs with imagined observations. Such updated beliefs can allow them to make more informed decisions, without necessitating the physical exploration of the world at all times. To achieve this human-like ability, we introduce the Generative World Explorer (Genex), an egocentric world exploration framework that allows an agent to mentally explore a large-scale 3D world (e.g., urban scenes) and acquire imagined observations to update its belief. This updated belief will then help the agent to make a more informed decision at the current step. To train Genex, we create a synthetic urban scene dataset, Genex-DB. Our experimental results demonstrate that (1) Genex can generate high-quality and consistent observations during long-horizon exploration of a large virtual physical world and (2) the beliefs updated with the generated observations can inform an existing decision-making model (e.g., an LLM agent) to make better plans.

Generative World Explorer

Load More... Subscribe

Johns Hopkins University, Whiting School of Engineering

Center for Language and Speech Processing
Hackerman 226
3400 North Charles Street, Baltimore, MD 21218-2680

(410) 516-4237
[email protected]

Whiting School of Engineering
Johns Hopkins University

Privacy Statement
Legal

© 2024 Johns Hopkins University

Center for Language and Speech Processing