Home
About
People
Publications
Seminars
- Seminar Calendar
- Seminar Videos
Workshops
HLT Masters
Resources

WS’03 Summer Workshop

June 1, 2003 @ 8:00 AM – August 1, 2003 @ 5:00 PM

Research Groups

Confidence Estimation for Natural Language Applications
Semantic Analysis Over Sparse Data
Syntax for Statistical Machine Translation

Subscribe to the CLSP Seminars Mailing List

Upcoming Seminars

12:00 pm Craig Greenberg (National Instit... @ Hackerman Hall B17

Craig Greenberg (National Instit... @ Hackerman Hall B17

Apr 22 @ 12:00 pm – 1:15 pm

Abstract The successes of speech and language technology research in recent years has led to the deployment of human language technology (HLT) in an increasing number of high-visibility, high-consequence applications. Well-established approaches to evaluating HLT[...]

12:00 pm Yulia Tsvetkov (University of Wa... @ Hackerman Hall B17

Yulia Tsvetkov (University of Wa... @ Hackerman Hall B17

Apr 26 @ 12:00 pm – 1:15 pm

Abstract Large language models (LMs) are pretrained on diverse data sources—news, discussion forums, books, online encyclopedias. A significant portion of this data includes facts and opinions which, on one hand, celebrate democracy and diversity of[...]

Add

Add to Timely Calendar
Add to Google
Add to Outlook
Add to Apple Calendar
Add to other calendar
Export to XML

Seminar Videos

Abstract: High-quality datasets are crucial for improving the capabilities and training efficiency of large language models. However, current datasets are typically prepared in an ad hoc, heuristic way. In this talk, Sang Michael Xie will present principled approaches to improving and understanding language models centered on the pre-training data distribution. First, he will describe how to improve the efficiency of training multipurpose language models by optimizing the mixture of data sources with robust optimization. Second, he will discuss an efficient importance resampling method for selecting relevant data from trillion-token-scale web datasets for training a specialized model. Finally, he will introduce a first theoretical analysis of in-context learning, a key capability of language models to learn from examples in a textual prompt, that traces the capability back to modeling coherence structure in the pre-training data.

Speaker Biography: Sang Michael Xie is a computer science PhD student at Stanford University advised by Percy Liang and Tengyu Ma. His research focuses on data-centric machine learning for language models, understanding pre-training and adaptation, and pre-training and self-training methods for robust machine learning. Xie was awarded a NDSEG Fellowship and was previously a student researcher at Google Brain. His work has been recognized as one of Scientific American‘s World-Changing Ideas, published in flagship venues such as Science, and covered by media outlets including The New York Times, The Washington Post, Reuters, BBC News, IEEE Spectrum, and The Verge.

Data-distributional Approaches for Generalizable Language Models -- Sang Michael Xie (Stanford)

UBM based Acoustic Modeling for ASR (Daniel Povey) - 2009

Abstract:
I discuss the application of Foundation Models in Astronomy through the collaborative efforts of the UniverseTBD consortium with a mission to democratize Science for everyone. One of our key objectives is to overcome the limitations of general-purpose Foundation Models, such as producing limited information in specialized fields. To this end, we have developed the first specialized large language model for Astronomy, AstroLLaMa-1. This model, enhanced by exposure to domain-specific literature from the NASA Astrophysics Data System and ArXiv, demonstrates improved text completion and embedding capabilities over existent GPT models. I further discuss the potential of LLMs in generating complex scientific hypotheses and extracting meaningful insights from astronomy literature. Our findings, validated by human experts, demonstrate the LLM capability in informed scientific critique and uncover intriguing patterns in the embedding space, highlighting the potential of LLMs to augment scientific inquiry. I will also discuss preliminary work with the multi-modal model AstroLLaVA, which allows us to interact with astronomical images via natural language. Through the work of UniverseTBD, we aim to explore how artificial intelligence can assist human intelligence in Astronomy and, more broadly, Science.

Bio:
Ioana Ciucă, who goes by Jo, is an interdisciplinary Jubilee Joint Fellow at the Australian National University, working across the School of Computing and the Research School of Astronomy & Astrophysics. Before joining ANU, Jo finished her PhD in Astrophysics at University College London in the United Kingdom, where she worked at the intersection of Astronomy and Machine Learning to understand the formation and evolution history of our Galaxy, the Milky Way. Jo is now focusing on utilizing foundation models that benefit researchers everywhere, working alongside the UniverseTBD team of more than 30 astronomers, engineers, ML practitioners and enthusiasts worldwide.

A Universe To Be Decided: Towards Specialized Foundation Models for Advancing Astronomy

Abstract:
Large language models like ChatGPT have shown extraordinary abilities for writing. While impressive at first glance, large language models aren't perfect and often make mistakes humans would not make. The main architecture behind ChatGPT mostly doesn't differ from early neural networks, and as a consequence, carries some of the same limitations. My work revolves around the use of neural networks like ChatGPT mixed with symbolic methods from early AI and how these two families of methods can combine to create more robust AI. I talk about some of the neurosymbolic methods I used for applications in story generation and understanding -- with the goal of eventually creating AI that can play Dungeons & Dragons. I also discuss pain points that I found for improving accessible communication and show how large language models can supplement such communication.

Bio:
Dr. Lara J. Martin (she/they) is an assistant professor at the University of Maryland, Baltimore County in the CSEE department, researching human-centered artificial intelligence with a focus on natural language processing applications. They have worked in the areas of automated story generation, augmentative and alternative communication (AAC) tools, AI for tabletop roleplaying games, speech processing, and affective computing—publishing in top-tier conferences such as AAAI, ACL, EMNLP, and IJCAI. They have also been featured in Wired and BBC Science Focus magazine.
Previously, Dr. Martin was a 2020 Computing Innovation Fellow (CIFellow) postdoctoral researcher at the University of Pennsylvania working with Dr. Chris Callison-Burch. She earned her PhD in Human-Centered Computing from the Georgia Institute of Technology, where she worked with Dr. Mark Riedl. She also has a MS in Language Technologies from Carnegie Mellon University and a BS in Computer Science & Linguistics from Rutgers University—New Brunswick.

Neurosymbolic AI or: How I Learned to Stop Worrying and Love the Large Language Model

Intro to information retrieval (James Mayfield) - 2009

Load More... Subscribe

Johns Hopkins University, Whiting School of Engineering

Center for Language and Speech Processing
Hackerman 226
3400 North Charles Street, Baltimore, MD 21218-2680

(410) 516-4237
[email protected]

Whiting School of Engineering
Johns Hopkins University

Privacy Statement
Legal

© 2024 Johns Hopkins University

Center for Language and Speech Processing