Home
About
People
Publications
Seminars
- Seminar Calendar
- Seminar Videos
Workshops
HLT Masters
Resources

Spring Break – No CLSP Seminar

Add to Calendar

Add to Timely Calendar
Add to Google
Add to Outlook
Add to Apple Calendar
Add to other calendar
Export to XML

When:

March 18, 2024 @ 12:00 pm – 1:15 pm

2024-03-18T12:00:00-04:00

2024-03-18T13:15:00-04:00

Seminar Videos

Title: CA-SSLR: Condition-Aware Self-Supervised Learning Representation for Generalized Speech Processing (NeurIPS 2024)

Speaker: Yen-ju Lu
https://neillu23.github.io/

Paper:
https://arxiv.org/abs/2412.04425

Abstract:
We introduce Condition-Aware Self-Supervised Learning Representation (CA-SSLR), a generalist conditioning model broadly applicable to various speech-processing tasks. Compared to standard fine-tuning methods that optimize for downstream models, CA-SSLR integrates language and speaker embeddings from earlier layers, making the SSL model aware of the current language and speaker context. This approach reduces the reliance on input audio features while preserving the integrity of the base SSLR. CA-SSLR improves the model's capabilities and demonstrates its generality on unseen tasks with minimal task-specific tuning. Our method employs linear modulation to dynamically adjust internal representations, enabling fine-grained adaptability without significantly altering the original model behavior. Experiments show that CA-SSLR reduces the number of trainable parameters, mitigates overfitting, and excels in under-resourced and unseen tasks. Specifically, CA-SSLR achieves a 10% relative reduction in LID errors, a 37% improvement in ASR CER on the ML-SUPERB benchmark, and a 27% decrease in SV EER on VoxCeleb-1, demonstrating its effectiveness.

CA-SSLR: Condition-Aware Self-Supervised Learning Representation for Generalized Speech Processing

Chenhao Tan (UChicago):
https://chenhaot.com/
https://cs.uchicago.edu/people/chenhao-tan/

Abstract
Human preferences have been the main component in guiding pretrained AI. In this talk, I argue that human preferences are often insufficient for complementing human intelligence and demonstrate the key role of human goals with two examples. First, hypothesis generation is critical for scientific discoveries. Instead of removing hallucinations, I will leverage data and labels as a guide to lead hallucination towards effective hypotheses. Second, I will use human perception as a guide for developing case-based explanations to support AI-assisted decision making. In both cases, faithfulness is “compromised” for achieving human goals. Finally, I will briefly discuss the case of identifying bias mechanisms in AI, where faithfulness is crucial. I will conclude with future directions towards complementary AI.

Bio
Tan is an assistant professor of computer science and data science at the University of Chicago, and is also affiliated with the Harris School of Public Policy. He obtained his PhD degree in the Department of Computer Science at Cornell University and bachelor’s degrees in computer science and in economics from Tsinghua University. Prior to joining the University of Chicago, he was an assistant professor at the University of Colorado Boulder and a postdoc at the University of Washington. His research interests include human-centered AI, natural language processing, and computational social science. His work has been covered by many news media outlets, such as the New York Times and the Washington Post. He also won a Sloan research fellowship, an NSF CAREER award, an NSF CRII award, a Google research scholar award, research awards from Amazon, IBM, JP Morgan, and Salesforce, a Facebook fellowship, and a Yahoo! Key Scientific Challenges award.

Beyond Human Preferences: Use Human Goals to Guide AI towards Complementary AI -- Chenhao Tan

Speaker: Jieyu Zhao (USC)
https://jyzhao.net/

Abstract: The rapid advancement of large language models (LLMs) has unlocked a myriad of possibilities for positive societal impact, ranging from enhancing accessibility and communication to supporting disaster response and public health initiatives. However, the deployment of these technologies also raises critical concerns regarding accountability, fairness, transparency, and ethical use. In this talk, I will discuss our efforts for auditing models, detecting and mitigating biases, and understanding how LLMs make decisions. We hope to open the conversation to foster a community-wide effort towards more accountable and inclusive LLM practices.

Trustworthy LLMs — our efforts on mitigating issues regarding social bias, safety and reliability

Abstract
Recent years saw the rise of multilingual language models that achieve high levels of performance for a large number of tasks, with some of them handling hundreds of languages at once. Premodern languages are usually underrepresented in such models, leading to poor performance in downstream applications. In my talk, I will introduce the Dharmamitra project, which aims to develop a diverse set of language models to address these shortcomings for the classical Asian low-resource languages Sanskrit, Tibetan, Classical Chinese, and Pali. These models are providing solutions for low-level NLP tasks such as word segmentation, morpho-syntactic tagging etc., as well as high-level tasks such as semantic search, machine translation, and general chatbot interaction. I will talk about the individual challenges and unique characteristics of the data involved, and what strategies we deploy to address these. I will also demonstrate how these different tools can be combined in an application that goes beyond simple sentence-to-sentence machine translation, but instead provides detailed grammatical explanations and corpus-wide search to provide the users with as much relevant information as possible. This application is helpful for early-stage languages learners on the one hand, as well as experienced researchers with high level of language knowledge and very specific demands on the other hand.

Biography
Sebastian Nehrdich is a research assistant at the University of California, Berkeley, where he is leading the MITRA project together with Prof. Kurt Keutzer. He is also currently completing a PhD degree in computational linguistics at the University of Duesseldorf, Germany. His main research interest is the application of NLP methods to premodern Asian languages with focus on Sanskrit, Pali, Tibetan and Classical Chinese. He holds a master’s degree in Buddhist Studies and worked extensively with literature in premodern languages while also publishing articles on NLP problems for these languages in major conferences. Since 2018, he is the chief developer of the BuddhaNexus platform, a website that provides an interactive and user-friendly interface for the exploration of textual reuse instances in large text collections of the Buddhist tradition with over a hundred million of intertextual links. Since 2023 The MITRA project provides grammatical analyzers, machine translation, semantic search capabilities for classical Asian languages and currently serves between 500-1000 users with primarily academic background on a daily basis.

MITRA: Beyond Just Machine Translation for Premodern Asian Low Resource Languages-Sebastian Nehrdich

Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements (Jack Zhang)

Jingyu Jack Zhang: https://jackz.io/

Paper: https://arxiv.org/abs/2410.08968

Abstract: The current paradigm for safety alignment of large language models (LLMs) follows a one-size-fits-all approach: the model refuses to interact with any content deemed unsafe by the model provider. This approach lacks flexibility in the face of varying social norms across cultures and regions. In addition, users may have diverse safety needs, making a model with static safety standards too restrictive to be useful, as well as too costly to be re-aligned. We propose Controllable Safety Alignment (CoSA), a framework designed to adapt models to diverse safety requirements without re-training. Instead of aligning a fixed model, we align models to follow safety configs -- free-form natural language descriptions of the desired safety behaviors -- that are provided as part of the system prompt. To adjust model safety behavior, authorized users only need to modify such safety configs at inference time. To enable that, we propose CoSAlign, a data-centric method for aligning LLMs to easily adapt to diverse safety configs. Furthermore, we devise a novel controllability evaluation protocol that considers both helpfulness and configured safety, summarizing them into CoSA-Score, and construct CoSApien, a human-authored benchmark that consists of real-world LLM use cases with diverse safety requirements and corresponding evaluation prompts. We show that CoSAlign leads to substantial gains of controllability over strong baselines including in-context alignment. Our framework encourages better representation and adaptation to pluralistic human values in LLMs, and thereby increasing their practicality.

Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements (Jack Zhang)

Load More... Subscribe

Johns Hopkins University, Whiting School of Engineering

Center for Language and Speech Processing
Hackerman 226
3400 North Charles Street, Baltimore, MD 21218-2680

(410) 516-4237
[email protected]

Whiting School of Engineering
Johns Hopkins University

Privacy Statement
Legal

© 2025 Johns Hopkins University

Center for Language and Speech Processing