Home
About
People
Publications
Seminars
- Seminar Calendar
- Seminar Videos
Workshops
HLT Masters
Resources

Seminars

Categories

Seminars

Student Seminars

Workshops
Tags
2016 2016 December Mesgarani 2017 2017 Bansal May 2017 Chelba April 2017 Jamieson April 2017 March Gannot 2017 Satheesh April 2017 Stevenson April 2018 2019 2020 2021 2022 2023 2024 2024 Chang 2024 Mortensen 2025 Abend Alwan Anastasopoulos Anumanchipalli April April Foreman APSA Roundtable August Auli Aziz Bafna Bai Balasubramanian Bamman Barker Belyy Bernstein Bhati Biadsy and Kanevsky Blair-Stanek Blasi Bonial Bosselut Boyd-Graber Brian Roark Broderick Brummer Busso Byerly Center for Language and Speech Processing Cervone Chaparala Chen Cherry Chiang Chintala Chiori Hori Cho Choi Christensen Chu-Carroll Ciuca CLSP CMU Cohen Computer Science Cristia December Deep Learning Diab Dietz DiPersio Don P. Giddens Inaugural Professorial Lecture Dredze Du Elbayad Ellis Ettinger Etzioni Fan Favaro February Februray Field Foulds Frederico Frermann Fried Fyshe Gabriel Gannot Gao Gardner Georgiou Goharian Gokaslan Goldwater Gong Google AI Google Research Gordon Gorman Graca Grant Green Greenberg Grosse Guan Guillermo Cecchi Gurevych Guzman Hajishirzi Hakkani-Tur Hansen Harrigian Harrison Hasegawa-Johnson Hashimoto He Heafield Heald Heck Hershey Hoffman Hopcroft Hou Hovy Hsu Huang Hussein Hutter Hwang Iyyer Jacek Mostwin Jahan Jain and Tensmeyer January Jason Eisner Ji Jiang Johns Hopkins University Joshi Joty Kamper Kann Karakos Kayser Khan Khashabi Kim Kirov Knill Kriz Kriz & Sanders Kyunghyun Cho Lake Lapata Lee Levy Lewis Li Li Xinyuan Likhomanenko Linzen Lipton Litman Livescu Louis Lozano-Diez Lu Maiti Mandel Marasovic March Markus Muller Martin Mathias May McCoy Mekyska Metze Mi Mihalcea Miloš Cerňak Mimno Minematsu Mitchell Mitra Mohamed Molybog Morency Moreno Mower-Provost Mueller Murray Mysore Nakov Narasimhan Natural Language Modeling Natural Language Processing Nehrdich Neural Machine Translation New York University November October Ostendorf Ouyang Parada Passonneau Peng Peterson Pinter Piper Prabhavalkar Prud'hommeaux Pustejovsky Qin Radinsky Raj Ramabhadran Ramesh Rashidi Rasooli Rastrow Re Rebecca Resnik Rene Vidal Rho Ringger Robinson Romanization Rose Rudinger Rush Rybicka Sadhu Saenko Sainath Sanders Schneider September Sha Sharma Shinji Watanabe Singh Siskind Sisman Slaney Smaragdis Smith Socher Soldaini Solorio Song Sproat Stanek Stanojevic Stent Steorts Strubell Suhr Summer School Sun Szymanski Tan Third Frederick Jelinek Memorial Summer Workshop Town Hall Tsvetkov Ture Velasquez Verma Viertanen Wang Weller White Whiting School of Engineering Wicks Wipperman ws12plenary ws14 ws14clamr ws14mt ws14participant ws14prelim ws14semantics ws16plenary Wu Xu Yang Yarmohammadi Yatskar Youn Yulia Tsvetkov Zelasko Zeldes Zellers Zettlemoyer Zhang Zhao Zhao & Deng Zhou Zhu

2024 Mar April 2025 Apr 2026

Sun	Mon	Tue	Wed	Thu	Fri	Sat
		1	2	3	4	5
6	7	8	9	10	11 Ting-Hao “Kenneth” Huang (PENN STATE) – “What Roles Will Humans Play in the Future of Data Annotation?” 12:00 pm Ting-Hao “Kenneth” Huang (PENN STATE) – “What Roles Will Humans Play in the Future of Data Annotation?” @ Hackerman Hall B17 Apr 11 @ 12:00 pm – 1:15 pm Tickets Abstract Large language models (LLMs) have rapidly and impressively taken over many tasks once handled by human annotators in constructing text datasets. But does this mean we no longer need humans in the annotation loop?[...]	12
13	14	15	16	17	18	19
20	21	22	23	24	25	26
27	28	29	30

2024 Mar April 2025 Apr 2026

Subscribe to filtered calendar

Add to Timely Calendar
Add to Google
Add to Outlook
Add to Apple Calendar
Add to other calendar
Export to XML

Seminars

Seminar Calendar
Seminar Videos

Subscribe to the CLSP Seminars Mailing List

Seminar Videos

Tianjian Li: https://tianjianl.github.io/

Related paper: https://arxiv.org/abs/2410.04579

Abstract: Data abundance across different domains exhibits a long-tailed distribution: few domains have abundant data, while most face data scarcity. Our work focuses on a multilingual setting, where available data is heavily skewed towards high-resource languages. Two common strategies to address this disparity are upsampling low-resource data (Temperature Sampling) and upweighting low-resource loss (Scalarization). These methods are often assumed to be equivalent, but this equivalence has not been rigorously established, prompting our investigation.
Through theoretical and empirical analysis, we identify when these two methods are equivalent and when they diverge. We prove that they are equivalent under full gradient descent but differ under stochastic gradient descent due to differences in gradient variance. Specifically, Temperature Sampling exhibits lower variance in gradient estimation compared to Scalarization, leading to faster convergence but a higher risk of overfitting. Based on these insights, we propose Cooldown, a strategy that starts by heavily upsampling low-resource languages to accelerate convergence and gradually reduces the upsampling to prevent overfitting -- achieving the best of both worlds. Our method competes effectively with existing data re-weighting techniques while offering computational efficiency.

Upsample or Upweight? Balanced Training on Heavily Imbalanced Datasets -- NAACL 2025

Based on the paper: https://arxiv.org/abs/2404.03862
Speaker: Jingyu (Jack) Zhang https://jackz.io/
Abstract:
To trust the fluent generations of large language models (LLMs), humans must be able to verify their correctness against trusted, external sources. Recent efforts, such as providing citations via retrieved documents or post-hoc provenance, enhance verifiability but provide no guarantees on their correctness. To address these limitations, we tackle the verifiability goal with a different philosophy: trivializing the verification process by developing models that quote verbatim statements from trusted sources in their pre-training data. We propose Quote-Tuning, which demonstrates the feasibility of aligning models to quote. The core of Quote-Tuning is a fast membership inference function that efficiently verifies text against trusted corpora. We leverage this tool to design a reward function to quantify quotes in model responses, and curate datasets for preference learning. Experiments show that Quote-Tuning significantly increases verbatim quotes from high-quality documents by up to 130% relative to base models while maintaining response quality. Quote-Tuning is applicable in different tasks, generalizes to out-of-domain data and diverse model families, and provides additional benefits to truthfulness. Our method not only serves as a hassle-free method to increase quoting but also opens up avenues for improving LLM trustworthiness through better verifiability.

Verifiable by Design: Aligning Language Models to Quote from Pre-Training Data (NAACL 2025)

The current paradigm for safety alignment of large language models (LLMs) follows a one-size-fits-all approach: the model refuses to interact with any content deemed unsafe by the model provider. This approach lacks flexibility in the face of varying social norms across cultures and regions. In addition, users may have diverse safety needs, making a model with static safety standards too restrictive to be useful, as well as too costly to be re-aligned.
We propose Controllable Safety Alignment (CoSA), a framework designed to adapt models to diverse safety requirements without re-training. Instead of aligning a fixed model, we align models to follow safety configs -- free-form natural language descriptions of the desired safety behaviors -- that are provided as part of the system prompt. To adjust model safety behavior, authorized users only need to modify such safety configs at inference time. To enable that, we propose CoSAlign, a data-centric method for aligning LLMs to easily adapt to diverse safety configs. Furthermore, we devise a novel controllability evaluation protocol that considers both helpfulness and configured safety, summarizing them into CoSA-Score, and construct CoSApien, a human-authored benchmark that consists of real-world LLM use cases with diverse safety requirements and corresponding evaluation prompts. We show that CoSAlign leads to substantial gains of controllability over strong baselines including in-context alignment. Our framework encourages better representation and adaptation to pluralistic human values in LLMs, and thereby increasing their practicality.

Speaker: Jack Jingyu Zhang
Website: https://jackz.io/
Paper link: https://arxiv.org/abs/2410.08968

Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements (ICLR 2025)

https://github.com/JHU-CLSP/turking-bench
https://turkingbench.github.io/

Abstract: Can advanced multi-modal models effectively tackle complex web-based tasks? Such tasks are often found on crowdsourcing platforms, where crowdworkers engage in challenging micro-tasks within web-based environments. Building on this idea, we present TurkingBench, a benchmark consisting of tasks presented as web pages with textual instructions and multi-modal contexts. Unlike previous approaches that rely on artificially synthesized web pages, our benchmark uses natural HTML pages originally designed for crowdsourcing workers to perform various annotation tasks. Each task's HTML instructions are instantiated with different values derived from crowdsourcing tasks, creating diverse instances. This benchmark includes 32.2K instances spread across 158 tasks. To support the evaluation of TurkingBench, we have developed a framework that links chatbot responses to actions on web pages (e.g., modifying a text box, selecting a radio button). We assess the performance of cutting-edge private and open-source models, including language-only and vision-language models (such as GPT4 and InternVL), on this benchmark. Our results show that while these models outperform random chance, there is still significant room for improvement. We hope that this benchmark will drive progress in the evaluation and development of web-based agents.

TurkingBench: A Challenge Benchmark for Web Agents --- NAACL 2025

Jack Jingyu Zhang: https://jackz.io/

Based on the following paper: https://arxiv.org/abs/2410.08968

Abstract: The current paradigm for safety alignment of large language models (LLMs) follows a one-size-fits-all approach: the model refuses to interact with any content deemed unsafe by the model provider. This approach lacks flexibility in the face of varying social norms across cultures and regions. In addition, users may have diverse safety needs, making a model with static safety standards too restrictive to be useful, as well as too costly to be re-aligned. We propose Controllable Safety Alignment (CoSA), a framework designed to adapt models to diverse safety requirements without re-training. Instead of aligning a fixed model, we align models to follow safety configs -- free-form natural language descriptions of the desired safety behaviors -- that are provided as part of the system prompt. To adjust model safety behavior, authorized users only need to modify such safety configs at inference time. To enable that, we propose CoSAlign, a data-centric method for aligning LLMs to easily adapt to diverse safety configs. Furthermore, we devise a novel controllability evaluation protocol that considers both helpfulness and configured safety, summarizing them into CoSA-Score, and construct CoSApien, a human-authored benchmark that consists of real-world LLM use cases with diverse safety requirements and corresponding evaluation prompts. We show that CoSAlign leads to substantial gains of controllability over strong baselines including in-context alignment. Our framework encourages better representation and adaptation to pluralistic human values in LLMs, and thereby increasing their practicality.

Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements -- ICLR 2025

Load More... Subscribe

Whiting School of Engineering
Johns Hopkins University

Privacy Statement
Legal

© 2025 Johns Hopkins University

Center for Language and Speech Processing