Home
About
People
Publications
Seminars
- Seminar Calendar
- Seminar Videos
Workshops
HLT Masters
Resources

Seminars

Categories

Seminars

Student Seminars

Workshops
Tags
2016 2016 December Mesgarani 2017 2017 Bansal May 2017 Chelba April 2017 Jamieson April 2017 March Gannot 2017 Satheesh April 2017 Stevenson April 2018 2019 2020 2021 2022 2023 2024 Abend Alwan Anastasopoulos Anumanchipalli April April Foreman APSA Roundtable August Auli Aziz Balasubramanian Bamman Barker Belyy Bhati Biadsy and Kanevsky Blair-Stanek Blasi Bonial Bosselut Boyd-Graber Brian Roark Broderick Brummer Busso Center for Language and Speech Processing Cervone Chen Cherry Chiang Chintala Chiori Hori Cho Choi Christensen Chu-Carroll Ciuca CLSP CMU Cohen Computer Science Cristia December Deep Learning Diab Dietz DiPersio Don P. Giddens Inaugural Professorial Lecture Dredze Ellis Ettinger Fan Favaro February Februray Field Foulds Frederico Frermann Fried Fyshe Gabriel Gannot Gao Gardner Georgiou Goharian Goldwater Gong Google AI Google Research Gordon Gorman Graca Grant Green Greenberg Guan Guillermo Cecchi Gurevych Guzman Hajishirzi Hakkani-Tur Hansen Harrigian Harrison Hasegawa-Johnson He Heafield Heald Heck Hershey Hoffman Hopcroft Hovy Hsu Huang Hussein Hutter Hwang Iyyer Jacek Mostwin Jahan Jain and Tensmeyer January Jason Eisner Ji Jiang Johns Hopkins University Joshi Joty Kamper Kann Karakos Kayser Khan Khashabi Kim Kirov Knill Kriz Kyunghyun Cho Lake Lapata Lee Levy Li Linzen Lipton Litman Livescu Louis Lozano-Diez Lu Maiti Mandel Marasovic March Markus Muller Martin Mathias May McCoy Mekyska Metze Mi Mihalcea Miloš Cerňak Mimno Minematsu Mitchell Mitra Mohamed Morency Moreno Mower-Provost Mueller Mysore Nakov Narasimhan Natural Language Modeling Natural Language Processing Neural Machine Translation New York University November October Ostendorf Ouyang Parada Passonneau Peng Peterson Pinter Piper Prabhavalkar Prud'hommeaux Pustejovsky Qin Radinsky Raj Ramabhadran Rasooli Rastrow Re Rebecca Resnik Rene Vidal Rho Ringger Romanization Rose Rudinger Rush Sadhu Saenko Sainath Sanders Schneider September Sha Shinji Watanabe Singh Siskind Sisman Slaney Smaragdis Smith Socher Solorio Song Sproat Stent Steorts Strubell Suhr Summer School Sun Szymanski Tan Third Frederick Jelinek Memorial Summer Workshop Town Hall Tsvetkov Ture Velasquez Verma Viertanen Wang White Whiting School of Engineering Wicks Wipperman ws12plenary ws14 ws14clamr ws14mt ws14participant ws14prelim ws14semantics ws16plenary Wu Xu Yang Yarmohammadi Yatskar Youn Yulia Tsvetkov Zelasko Zeldes Zellers Zettlemoyer Zhao Zhou Zhu

2023 Mar April 2024 May 2025

Sun	Mon	Tue	Wed	Thu	Fri	Sat
	1	2	3	4	5	6
7	8	9	10	11	12	13
14	15	16	17	18	19	20
21	22	23	24	25	26 Yulia Tsvetkov (University of Washington) “LLMs under the Microscope: Illuminating the Blind Spots and Improving the Reliability of Language Models” 12:00 pm Yulia Tsvetkov (University of Washington) “LLMs under the Microscope: Illuminating the Blind Spots and Improving the Reliability of Language Models” @ Hackerman Hall B17 Apr 26 @ 12:00 pm – 1:15 pm Tickets Abstract Large language models (LMs) are pretrained on diverse data sources—news, discussion forums, books, online encyclopedias. A significant portion of this data includes facts and opinions which, on one hand, celebrate democracy and diversity of[...]	27
28	29	30

2023 Mar April 2024 May 2025

Subscribe to filtered calendar

Add to Timely Calendar
Add to Google
Add to Outlook
Add to Apple Calendar
Add to other calendar
Export to XML

Seminars

Seminar Calendar
Seminar Videos

Subscribe to the CLSP Seminars Mailing List

Seminar Videos

Abstract
The rapidity with which children acquire language is one of the mysteries of human cognition. A widely held view is that children master language by means of a language-specific learning device. An earlier proposal, generating renewed interest, is that children make use of domain-general, associative learning mechanisms in acquiring language. However, we know little about the actual learning mechanisms involved, making it difficult to determine the relative contributions of innate and acquired knowledge. A recent approach to studying this problem exposes infants to artificial languages and assesses the resulting learning. Studies using this paradigm have led to a number of exciting discoveries regarding the learning mechanisms available during infancy. This approach is useful for achieving finer-grained characterizations of infant learning mechanisms than previously imagined. Such characterizations should result in a better understanding of the relative contributions of innate and learned factors to language acquisition, as well as the dynamic between the two. I will present several studies using this methodology and will discuss the implications for the study of language acquisition.

Artificial Language Learning and Language Acquisition – Rebecca Gomez (JHU) - 2000

ICLR 2024
https://arxiv.org/abs/2310.00840

Abstract:
Text generation models are notoriously vulnerable to errors in the training data. With the wide-spread availability of massive amounts of web-crawled data becoming more commonplace, how can we enhance the robustness of models trained on a massive amount of noisy web-crawled text? In our work, we propose Error Norm Truncation (ENT), a robust enhancement method to the standard training objective that truncates noisy data. Compared to methods that only uses the negative log-likelihood loss to estimate data quality, our method provides a more accurate estimation by considering the distribution of non-target tokens, which is often overlooked by previous work. Through comprehensive experiments across language modeling, machine translation, and text summarization, we show that equipping text generation models with ENT improves generation quality over standard training and previous soft and hard truncation methods. Furthermore, we show that our method improves the robustness of models against two of the most detrimental types of noise in machine translation, resulting in an increase of more than 2 BLEU points over the MLE baseline when up to 50% of noise is added to the data.

Error Norm Truncation: Robust Training in the Presence of Data Noise for Text Generation Models

Eric Fosler-Lussier: Speech Recognition - 2000

Lecture given as part of CS 601.471/671 NLP: Self-supervised Models:
https://self-supervised.cs.jhu.edu/sp2024/

Speaker: Tianjian Li:
https://tianjianl.github.io/

Tutorial: Efficient language models -- Tianjian Li (JHU)

Abstract: High-quality datasets are crucial for improving the capabilities and training efficiency of large language models. However, current datasets are typically prepared in an ad hoc, heuristic way. In this talk, Sang Michael Xie will present principled approaches to improving and understanding language models centered on the pre-training data distribution. First, he will describe how to improve the efficiency of training multipurpose language models by optimizing the mixture of data sources with robust optimization. Second, he will discuss an efficient importance resampling method for selecting relevant data from trillion-token-scale web datasets for training a specialized model. Finally, he will introduce a first theoretical analysis of in-context learning, a key capability of language models to learn from examples in a textual prompt, that traces the capability back to modeling coherence structure in the pre-training data.

Speaker Biography: Sang Michael Xie is a computer science PhD student at Stanford University advised by Percy Liang and Tengyu Ma. His research focuses on data-centric machine learning for language models, understanding pre-training and adaptation, and pre-training and self-training methods for robust machine learning. Xie was awarded a NDSEG Fellowship and was previously a student researcher at Google Brain. His work has been recognized as one of Scientific American‘s World-Changing Ideas, published in flagship venues such as Science, and covered by media outlets including The New York Times, The Washington Post, Reuters, BBC News, IEEE Spectrum, and The Verge.

Data-distributional Approaches for Generalizable Language Models -- Sang Michael Xie (Stanford)

Load More... Subscribe

Johns Hopkins University, Whiting School of Engineering

Center for Language and Speech Processing
Hackerman 226
3400 North Charles Street, Baltimore, MD 21218-2680

(410) 516-4237
[email protected]

Whiting School of Engineering
Johns Hopkins University

Privacy Statement
Legal

© 2024 Johns Hopkins University

Center for Language and Speech Processing