BEGIN:VCALENDAR VERSION:2.0 PRODID:-//128.220.36.25//NONSGML kigkonsult.se iCalcreator 2.26.9// CALSCALE:GREGORIAN METHOD:PUBLISH X-FROM-URL:https://www.clsp.jhu.edu X-WR-TIMEZONE:America/New_York BEGIN:VTIMEZONE TZID:America/New_York X-LIC-LOCATION:America/New_York BEGIN:STANDARD DTSTART:20231105T020000 TZOFFSETFROM:-0400 TZOFFSETTO:-0500 RDATE:20241103T020000 TZNAME:EST END:STANDARD BEGIN:DAYLIGHT DTSTART:20240310T020000 TZOFFSETFROM:-0500 TZOFFSETTO:-0400 RDATE:20250309T020000 TZNAME:EDT END:DAYLIGHT END:VTIMEZONE BEGIN:VEVENT UID:ai1ec-21041@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nNarration is a universal human practice that serves a s a key site of education\, collective memory\, fostering social belief sy stems\, and furthering human creativity. Recent studies in economics (Shil ler\, 2020)\, climate science (Bushell et al.\, 2017)\, political polariza tion (Kubin et al.\, 2021)\, and mental health (Adler et al.\, 2016) sugge st an emerging interdisciplinary consensus that narrative is a central con cept for understanding human behavior and beliefs. For close to half a cen tury\, the field of narratology has developed a rich set of theoretical fr ameworks for understanding narrative. And yet these theories have largely gone untested on large\, heterogenous collections of texts. Scholars conti nue to generate schemas by extrapolating from small numbers of manually ob served documents. In this talk\, I will discuss how we can use machine lea rning to develop data-driven theories of narration to better understand wh at Labov and Waletzky called “the simplest and most fundamental narrative structures.” How can machine learning help us approach what we might call a minimal theory of narrativity?\nBiography\nAndrew Piper is Professor and William Dawson Scholar in the Department of Languages\, Literatures\, and Cultures at McGill University. He is the director of _.txtlab \n_\,\n a l aboratory for cultural analytics\, and editor of the /Journal of Cultural Analytics/\, an open-access journal dedicated to the computational study o f culture. He is the author of numerous books and articles on the relation ship of technology and reading\, including /Book Was There: Reading in Ele ctronic Times/(Chicago 2012)\, /Enumerations: Data and Literary Study/(Chi cago 2018)\, and most recently\, /Can We Be Wrong? The Problem of Textual Evidence in a Time of Data/(Cambridge 2020). DTSTART;TZID=America/New_York:20211112T120000 DTEND;TZID=America/New_York:20211112T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Andrew Piper (McGill University) ” How can we use machine learning to understand narration?” URL:https://www.clsp.jhu.edu/events/andrew-piper-mcgill-university-how-can- we-use-machine-learning-to-understand-narration/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n
\\nAbstr act
\nNarration is a universal human practice that serves a s a key site of education\, collective memory\, fostering social belief sy stems\, and furthering human creativity. Recent studies in economics (Shil ler\, 2020)\, climate science (Bushell et al.\, 2017)\, political polariza tion (Kubin et al.\, 2021)\, and mental health (Adler et al.\, 2016) sugge st an emerging interdisciplinary consensus that narrative is a central con cept for understanding human behavior and beliefs. For close to half a cen tury\, the field of narratology has developed a rich set of theoretical fr ameworks for understanding narrative. And yet these theories have largely gone untested on large\, heterogenous collections of texts. Scholars conti nue to generate schemas by extrapolating from small numbers of manually ob served documents. In this talk\, I will discuss how we can use machine lea rning to develop data-driven theories of narration to better understand wh at Labov and Waletzky called “the simplest and most fundamental narrative structures.” How can machine learning help us approach what we might call a minimal theory of narrativity?
\nBiography
\n< p>Andrew Piper is Professor and William D awson Scholar in the Department of Languages\, Literatures\, and Cultures at McGill University. He is the director of _.txtlab \n\na laboratory for cultural ana lytics\, and editor of the /Journal of Cultural Analytics/\, an open-acces s journal dedicated to the computational study of culture. He is the autho r of numerous books and articles on the relationship of technology and rea ding\, including /Book Was There: Reading in Electronic Times/(Chicago 201 2)\, /Enumerations: Data and Literary Study/(Chicago 2018)\, and most rece ntly\, /Can We Be Wrong? The Problem of Textual Evidence in a Time of Data /(Cambridge 2020).
\n X-TAGS;LANGUAGE=en-US:2021\,November\,Piper END:VEVENT BEGIN:VEVENT UID:ai1ec-21494@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION:Abstract\nAdversarial attacks deceive neural network systems by adding carefully crafted perturbations to benign signals. Being almost im perceptible to humans\, these attacks pose a severe security threat to the state-of-the-art speech and speaker recognition systems\, making it vital to propose countermeasures against them. In this talk\, we focus on 1) cl assification of a given adversarial attack into attack algorithm type\, th reat model type\, and signal-to-adversarial-noise ratios\, 2) developing a novel speech denoising solution to further improve the classification per formance. \nOur proposed approach uses an x-vector network as a signature extractor to get embeddings\, which we call signatures. These signatures c ontain information about the attack and can help classify different attack algorithms\, threat models\, and signal-to-adversarial-noise ratios. We d emonstrate the transferability of such signatures to other tasks. In parti cular\, a signature extractor trained to classify attacks against speaker identification can also be used to classify attacks against speaker verifi cation and speech recognition. We also show that signatures can be used to detect unknown attacks i.e. attacks not included during training. Lastly \, we propose to improve the signature extractor by making the job of the signature extractor easier by removing the clean signal from the adversari al example (which consists of clean signal+perturbation). We train our sig nature extractor using adversarial perturbation. At inference time\, we us e a time-domain denoiser to obtain adversarial perturbation from adversari al examples. Using our improved approach\, we show that common attacks in the literature (Fast Gradient Sign Method (FGSM)\, Projected Gradient Desc ent (PGD)\, Carlini-Wagner (CW) ) can be classified with accuracy as high as 96%. We also detect unknown attacks with an equal error rate (EER) of a bout 9%\, which is very promising. DTSTART;TZID=America/New_York:20220304T120000 DTEND;TZID=America/New_York:20220304T131500 LOCATION:Ames Hall 234 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Student Seminar – Sonal Joshi “Classify and Detect Adversarial Atta cks Against Speaker and Speech Recognition Systems” URL:https://www.clsp.jhu.edu/events/student-seminar-sonal-joshi/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nAdversarial attacks deceive neural network systems by adding carefully crafted perturbations to benign signals. Being almost imperceptible to humans\, these attacks pose a severe security thr eat to the state-of-the-art speech and speaker recognition systems\, makin g it vital to propose countermeasures against them. In this talk\, we focu s on 1) classification of a given adversarial attack into attack algorithm type\, threat model type\, and signal-to-adversarial-noise ratios\, 2) de veloping a novel speech denoising solution to further improve the classifi cation performance.
\nOur proposed approach uses a n x-vector network as a signature extractor to get embeddings\, which we c all signatures. These signatures contain information about the attack and can help classify different attack algorithms\, threat models\, and signal -to-adversarial-noise ratios. We demonstrate the transferability of such s ignatures to other tasks. In particular\, a signature extractor trained to classify attacks against speaker identification can also be used to class ify attacks against speaker verification and speech recognition. We also s how that signatures can be used to detect unknown attacks i.e. attacks not included during training. Lastly\, we propose to improve the signature e xtractor by making the job of the signature extractor easier by removing t he clean signal from the adversarial example (which consists of clean sign al+perturbation). We train our signature extractor using adversarial pertu rbation. At inference time\, we use a time-domain denoiser to obtain adver sarial perturbation from adversarial examples. Using our improved approach \, we show that common attacks in the literature (Fast Gradient Sign Metho d (FGSM)\, Projected Gradient Descent (PGD)\, Carlini-Wagner (CW) ) can be classified with accuracy as high as 96%. We also detect unknown attacks w ith an equal error rate (EER) of about 9%\, which is very promising.
\n X-TAGS;LANGUAGE=en-US:2022\,Joshi\,March END:VEVENT BEGIN:VEVENT UID:ai1ec-23302@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION: DTSTART;TZID=America/New_York:20230130T120000 DTEND;TZID=America/New_York:20230130T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Daniel Fried (CMU) URL:https://www.clsp.jhu.edu/events/daniel-fried-cmu/ X-COST-TYPE:free X-TAGS;LANGUAGE=en-US:2023\,Fried\,January END:VEVENT BEGIN:VEVENT UID:ai1ec-23304@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nTransformers are essential to pretraining. As we appr oach 5 years of BERT\, the connection between attention as architecture an d transfer learning remains key to this central thread in NLP. Other archi tectures such as CNNs and RNNs have been used to replicate pretraining res ults\, but these either fail to reach the same accuracy or require supplem ental attention layers. This work revisits the semanal BERT result and con siders pretraining without attention. We consider replacing self-attention layers with recently developed approach for long-range sequence modeling and transformer architecture variants. Specifically\, inspired by recent p apers like the structured space space sequence model (S4)\, we use simple routing layers based on state-space models (SSM) and a bidirectional model architecture based on multiplicative gating. We discuss the results of th e proposed Bidirectional Gated SSM (BiGS) and present a range of analysis into its properties. Results show that architecture does seem to have a no table impact on downstream performance and a different inductive bias that is worth exploring further.\nBiography\nAlexander “Sasha” Rush is an Asso ciate Professor at Cornell Tech. His work is at the intersection of natura l language processing and generative modeling with applications in text ge neration\, efficient inference\, and controllability. He has written sever al popular open-source software projects supporting NLP research and data science\, and works part-time as a researcher at Hugging Face. He is the s ecretary of ICLR and developed software used to run virtual conferences du ring COVID. His work has received paper and demo awards at major NLP\, vis ualization\, and hardware conferences\, an NSF Career Award\, and a Sloan Fellowship. He tweets and blogs\, mostly about coding and ML\, at @srush_n lp. DTSTART;TZID=America/New_York:20230203T120000 DTEND;TZID=America/New_York:20230203T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Sasha Rush (Cornell University) “Pretraining Without Attention” URL:https://www.clsp.jhu.edu/events/sasha-rush-cornell-university/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nTransformers are essential to pretraining. As we appr oach 5 years of BERT\, the connection between attention as architecture an d transfer learning remains key to this central thread in NLP. Other archi tectures such as CNNs and RNNs have been used to replicate pretraining res ults\, but these either fail to reach the same accuracy or require supplem ental attention layers. This work revisits the semanal BERT result and con siders pretraining without attention. We consider replacing self-attention layers with recently developed approach for long-range sequence modeling and transformer architecture variants. Specifically\, inspired by recent p apers like the structured space space sequence model (S4)\, we use simple routing layers based on state-space models (SSM) and a bidirectional model architecture based on multiplicative gating. We discuss the results of th e proposed Bidirectional Gated SSM (BiGS) and present a range of analysis into its properties. Results show that architecture does seem to have a no table impact on downstream performance and a different inductive bias that is worth exploring further.
\nBiography
\nAbstr act
\nWhile large language models have advanced the state-o f-the-art in natural language processing\, these models are trained on lar ge-scale datasets\, which may include harmful information. Studies have sh own that as a result\, the models exhibit social biases and generate misin formation after training. In this talk\, I will discuss my work on analyzi ng and interpreting the risks of large language models across the areas of fairness\, trustworthiness\, and safety. I will first describe my researc h in the detection of dialect bias between African American English (AAE) vs. Standard American English (SAE). The second part investigates the trus tworthiness of models through the memorization and subsequent generation o f conspiracy theories. I will end my talk with recent work in AI safety re garding text that may lead to physical harm.
\nBiography
\nSharon is a 5th-year Ph.D. candidate at the University of Ca lifornia\, Santa Barbara\, where she is advised by Professor William Wang. Her research interests lie in natural language processing\, with a focus on Responsible AI. Sharon’s research spans the subareas of fairness\, trus tworthiness\, and safety\, with publications in ACL\, EMNLP\, WWW\, and LR EC. She has spent summers interning at AWS\, Meta\, and Pinterest. Sharon is a 2022 EECS Rising Star and a current recipient of the Amazon Alexa AI Fellowship for Responsible AI.
\n X-TAGS;LANGUAGE=en-US:2023\,February\,Levy END:VEVENT BEGIN:VEVENT UID:ai1ec-23308@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nBiases in datasets\, or unintentionally introduced sp urious cues\, are a common source of misspecification in machine learning. Performant models trained on such data can gender stereotype or be brittl e under distribution shift. In this talk\, we present several results in multimodal and question answering applications studying sources of dataset bias\, and several mitigation methods. We propose approaches where known dimensions of dataset bias are explicitly factored out of a model during learning\, without needing to modify data. Finally\, we ask whether datase t biases can be attributable to annotator behavior during annotation. Draw ing inspiration from work in psychology on cognitive biases\, we show cert ain behavioral patterns are highly indicative of the creation of problemat ic (but valid) data instances in question answering. We give evidence that many existing observations around how dataset bias propagates to models c an be attributed to data samples created by annotators we identify.\nBiogr aphy\nMark Yatskar is an Assistant Professor at University of Pennsylvania in the department of Computer and Information Science. He did his PhD at University of Washington co-advised by Luke Zettlemoyer and Ali Farhadi. H e was a Young Investigator at the Allen Institute for Artificial Intellige nce for several years working with their computer vision team\, Prior. His work spans Natural Language Processing\, Computer Vision\, and Fairness i n Machine Learning. He received a Best Paper Award at EMNLP for work on ge nder bias amplification\, and his work has been featured in Wired and the New York Times. DTSTART;TZID=America/New_York:20230210T120000 DTEND;TZID=America/New_York:20230210T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Mark Yatskar (University of Pennsylvania) “Understanding Dataset Bi ases: Behavioral Indicators During Annotation and Contrastive Mitigations” URL:https://www.clsp.jhu.edu/events/mark-yatskar-university-of-pennsylvania / X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nBiases in datasets\, or unintentionally introduced sp urious cues\, are a common source of misspecification in machine learning. Performant models trained on such data can gender stereotype or be brittl e under distribution shift. In this talk\, we present several results in multimodal and question answering applications studying sources of dataset bias\, and several mitigation methods. We propose approaches where known dimensions of dataset bias are explicitly factored out of a model during learning\, without needing to modify data. Finally\, we ask whether datase t biases can be attributable to annotator behavior during annotation. Draw ing inspiration from work in psychology on cognitive biases\, we show cert ain behavioral patterns are highly indicative of the creation of problemat ic (but valid) data instances in question answering. We give evidence that many existing observations around how dataset bias propagates to models c an be attributed to data samples created by annotators we identify.
\n< p>Biography\nMark Yatskar is an Assistan t Professor at University of Pennsylvania in the department of Computer an d Information Science. He did his PhD at University of Washington co-advis ed by Luke Zettlemoyer and Ali Farhadi. He was a Young Investigator at the Allen Institute for Artificial Intelligence for several years working wit h their computer vision team\, Prior. His work spans Natural Language Proc essing\, Computer Vision\, and Fairness in Machine Learning. He received a Best Paper Award at EMNLP for work on gender bias amplification\, and his work has been featured in Wired and the New York Times.
\n\n X-TAGS;LANGUAGE=en-US:2023\,February\,Yatskar END:VEVENT BEGIN:VEVENT UID:ai1ec-23314@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nWhile GPT models have shown impressive performance on summarization and open-ended text generation\, it’s important to assess t heir abilities on more constrained text generation tasks that require sign ificant and diverse rewritings. In this talk\, I will discuss the challeng es of evaluating systems that are highly competitive and perform close to humans on two such tasks: (i) paraphrase generation and (ii) text simplifi cation. To address these challenges\, we introduce an interactive Rank-and -Rate evaluation framework. Our results show that GPT-3.5 has made a major step up from fine-tuned T5 in paraphrase generation\, but still lacks the diversity and creativity of humans who spontaneously produce large quanti ties of paraphrases.\nAdditionally\, we demonstrate that GPT-3.5 performs similarly to a single human in text simplification\, which makes it diffic ult for existing automatic evaluation metrics to distinguish between the t wo. To overcome this shortcoming\, we propose LENS\, a learnable evaluatio n metric that outperforms SARI\, BERTScore\, and other existing methods in both automatic evaluation and minimal risk decoding for text generation. \nBiography\nWei Xu is an assistant professor in the School of Interactive Computing at the Georgia Institute of Technology\, where she is also affi liated with the new NSF AI CARING Institute and Machine Learning Center. S he received her Ph.D. in Computer Science from New York University and her B.S. and M.S. from Tsinghua University. Xu’s research interests are in na tural language processing\, machine learning\, and social media\, with a f ocus on text generation\, stylistics\, robustness and controllability of m achine learning models\, and reading and writing assistive technology. She is a recipient of the NSF CAREER Award\, CrowdFlower AI for Everyone Awar d\, Criteo Faculty Research Award\, and Best Paper Award at COLING’18. She has also received funds from DARPA and IARPA. She is an elected member of the NAACL executive board and regularly serves as a senior area chair for AI/NLP conferences. DTSTART;TZID=America/New_York:20230224T120000 DTEND;TZID=America/New_York:20230224T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Wei Xu (Georgia Tech) “GPT-3 vs Humans: Rethinking Evaluation of Na tural Language Generation” URL:https://www.clsp.jhu.edu/events/wei-xu-georgia-tech/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n
\\nAbstr act
\nWhile GPT mo dels have shown impressive performance on summarization and open-ended tex t generation\, it’s important to assess their abilities on more constraine d text generation tasks that require significant and diverse rewritings. I n this talk\, I will discuss the challenges of evaluating systems that are highly competitive and perform close to humans on two such tasks: (i) par aphrase generation and (ii) text simplification. To address these challeng es\, we introduce an interactive Rank-and-Rate evaluation framework. Our r esults show that GPT-3.5 has made a major step up from fine-tuned T5 in pa raphrase generation\, but still lacks the diversity and creativity of huma ns who spontaneously produce large quantities of paraphrases.
\nAdditionally\, we demon strate that GPT-3.5 performs similarly to a single human in text simplific ation\, which makes it difficult for existing automatic evaluation metrics to distinguish between the two. To overcome this shortcoming\, we propose LENS\, a learnable evaluation metric that outperforms SARI\, BERTScore\, and other existing methods in both automatic evaluation and minimal risk d ecoding for text generation.
\nBiography
\nWei Xu is an assis tant professor in the School of Interactive Computing at the Georgia Insti tute of Technology\, where she is also affiliated with the new NSF AI CARI NG Institute and Machine Learning Center. She received her Ph.D. in Comput er Science from New York University and her B.S. and M.S. from Tsinghua Un iversity. Xu’s research interests are in natural language processing\, mac hine learning\, and social media\, with a focus on text generation\, styli stics\, robustness and controllability of machine learning models\, and re ading and writing assistive technology. She is a recipient of the NSF CARE ER Award\, CrowdFlower AI for Everyone Award\, Criteo Faculty Research Awa rd\, and Best Paper Award at COLING’18. She has also received funds from D ARPA and IARPA. She is an elected member of the NAACL executive board and regularly serves as a senior area chair for AI/NLP conferences.
\n X-TAGS;LANGUAGE=en-US:2023\,February\,Xu END:VEVENT BEGIN:VEVENT UID:ai1ec-23316@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nUnderstanding the implications underlying a text is c ritical to assessing its impact\, in particular the social dynamics that m ay result from a reading of the text. This requires endowing artificial in telligence (AI) systems with pragmatic reasoning\, for example to correctl y conclude that the statement “Epidemics and cases of disease in the 21st century are “staged”” relates to unfounded conspiracy theories. In this ta lk\, I discuss how shortcomings in the ability of current AI systems to re ason about pragmatics present challenges to equitable detection of false o r harmful language. I demonstrate how these shortcomings can be addressed by imposing human-interpretable structure on deep learning architectures u sing insights from linguistics.In the first part of the talk\, I describe how adversarial text generation algorithms can be used to improve robustne ss of content moderation systems. I then introduce a pragmatic formalism f or reasoning about harmful implications conveyed by social media text. I s how how this pragmatic approach can be combined with generative neural lan guage models to uncover implications of news headlines. I also address the bottleneck to progress in text generation posed by gaps in evaluation of factuality. I conclude by showing how context-aware content moderation can be used to ensure safe interactions with conversational agents.\n \nBiogr aphy\nSaadia Gabriel is a PhD candidate in the Paul G. Allen School of Com puter Science & Engineering at the University of Washington\, advised by P rof. Yejin Choi and Prof. Franziska Roesner. Her researchrevolves around n atural language processing and machine learning\, with a particular focus on building systems for understanding how social commonsense manifests in text (i.e. how do people typically behave in social scenarios)\, as well a s mitigating spread of false or harmful text (e.g. Covid-19 misinformation ). Her work has been covered by a wide range of media outlets like Forbes and TechCrunch. It has also received a 2019 ACL best short paper nominatio n\, a 2019 IROS RoboCup best paper nomination and won a best paper award a t the 2020 WeCNLP summit. Prior to her PhD\, Saadia received a BA summa cu m laude from Mount Holyoke College in Computer Science and Mathematics.\n DTSTART;TZID=America/New_York:20230227T120000 DTEND;TZID=America/New_York:20230227T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Saadia Gabriel (University of Washington) “Socially Responsible and Factual Reasoning for Equitable AI Systems” URL:https://www.clsp.jhu.edu/events/saadia-gabriel-university-of-washington -socially-responsible-and-factual-reasoning-for-equitable-ai-systems/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nUnderstanding the implications underlying a text is c
ritical to assessing its impact\, in particular the social dynamics that m
ay result from a reading of the text. This requires endowing artificial in
telligence (AI) systems with pragmatic reasoning\, for example to correctl
y conclude that the statement “Epidemics and cases of disease in the 21st
century are “staged”” relates to unfounded conspiracy theories. In this ta
lk\, I discuss how shortcomings in the ability of current AI systems to re
ason about pragmatics present challenges to equitable detection of false o
r harmful language. I demonstrate how these shortcomings can be addressed
by imposing human-interpretable structure on deep learning architectures u
sing insights from linguistics.
In the first part of the talk\, I describe how adversarial text gen
eration algorithms can be used to improve robustness of content moderation
systems. I then introduce a pragmatic formalism for reasoning about harmf
ul implications conveyed by social media text. I show how this pragmatic a
pproach can be combined with generative neural language models to uncover
implications of news headlines. I also address the bottleneck to progress
in text generation posed by gaps in evaluation of factuality. I conclude b
y showing how context-aware content moderation can be used to ensure safe
interactions with conversational agents.
\n
Biography
\nSaadia Gabr iel is a PhD candidate in the Paul G. Allen School of Computer Scie nce & Engineering at the University of Washington\, advised by Prof. Yejin Choi and Prof. Franziska Roesner. Her research re volves around natural language processing and machine learning\, with a pa rticular focus on building systems for understanding how social commonsens e manifests in text (i.e. how do people typically behave in social scenari os)\, as well as mitigating spread of false or harmful text (e.g. Covid-19 misinformation). Her work has been covered by a wide range of media outle ts like Forbes and TechCrunch. It has also received a 2019 ACL best short paper nomination\, a 2019 IROS RoboCup best paper nomination and won a bes t paper award at the 2020 WeCNLP summit. Prior to her PhD\, Saadia received a BA summa cum laude from Mount Holyoke College in Computer Sc ience and Mathematics.
\n\n X-TAGS;LANGUAGE=en-US:2023\,February\,Gabriel END:VEVENT BEGIN:VEVENT UID:ai1ec-23320@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nSpeech communications represents a core domain for ed ucation\, team problem solving\, social engagement\, and business interact ions. The ability for Speech Technology to extract layers of knowledge and assess engagement content represents the next generation of advanced spee ch solutions. Today\, the emergence of BIG DATA\, Machine Learning\, as we ll as voice enabled speech systems have required the need for effective vo ice capture and automatic speech/speaker recognition. The ability to emplo y speech and language technology to assess human-to-human interactions off ers new research paradigms having profound impact on assessing human inter action. In this talk\, we will focus on big data naturalistic audio proces sing relating to (i) child learning spaces\, and (ii) the NASA APOLLO luna r missions. ML based technology advancements include automatic audio diari zation\, speech recognition\, and speaker recognition. Child-Teacher based assessment of conversational interactions are explored\, including keywor d and “WH-word” (e.g.\, who\, what\, etc.). Diarization processing solutio ns are applied to both classroom/learning space child speech\, as well as massive APOLLO data. CRSS-UTDallas is expanding our original Apollo-11 cor pus\, resulting in a massive multi-track audio processing challenge to mak e available 150\,000hrs of Apollo mission data to be shared with science c ommunities: (i) speech/language technology\, (ii) STEM/science and team-ba sed researchers\, and (iii) education/historical/archiving specialists. Ou r goals here are to provide resources which allow to better understand how people work/learn collaboratively together. For Apollo\, to accomplish on e of mankind’s greatest scientific/technological challenges in the last ce ntury.\nBiography\nJohn H.L. Hansen\, received Ph.D. & M.S. degrees from G eorgia Institute of Technology\, and B.S.E.E. from Rutgers Univ. He joined Univ. of Texas at Dallas (UTDallas) in 2005\, where he currently serves a s Associate Dean for Research\, Prof. of ECE\, Distinguished Univ. Chair i n Telecom. Engineering\, and directs Center for Robust Speech Systems (CRS S). He is an ISCA Fellow\, IEEE Fellow\, and has served as Member and TC-C hair of IEEE Signal Proc. Society\, Speech & Language Proc. Tech. Comm.(SL TC)\, and Technical Advisor to U.S. Delegate for NATO (IST/TG-01). He serv ed as ISCA President (2017-21)\, continues to serve on ISCA Board (2015-23 ) as Treasurer\, has supervised 99 PhD/MS thesis candidates (EE\,CE\,BME\, TE\,CS\,Ling.\,Cog.Sci.\,Spch.Sci.\,Hear.Sci)\, was recipient of 2020 UT-D allas Provost’s Award for Grad. PhD Research Mentoring\; author/co-author of 865 journal/conference papers including 14 textbooks in the field of sp eech/language/hearing processing & technology including coauthor of textbo ok Discrete-Time Processing of Speech Signals\, (IEEE Press\, 2000)\, and lead author of the report “The Impact of Speech Under ‘Stress’ on Military Speech Technology\,” (NATO RTO-TR-10\, 2000). He served as Organizer\, Ch air/Co-Chair/Tech.Chair for ISCA INTERSPEECH-2022\, IEEE ICASSP-2010\, IEE E SLT-2014\, ISCA INTERSPEECH-2002\, and Tech. Chair for IEEE ICASSP-2024. He received the 2022 IEEE Signal Processing Society Leo Beranek MERITORIO US SERVICE Award.\n DTSTART;TZID=America/New_York:20230303T120000 DTEND;TZID=America/New_York:20230303T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:John Hansen (University of Texas at Dallas) “Challenges and Advance ments in Speaker Diarization & Recognition for Naturalistic Data Streams” URL:https://www.clsp.jhu.edu/events/john-hansen-university-of-texas-at-dall as/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\n
Abstr act
\nSpeech communications represents a core domain for ed ucation\, team problem solving\, social engagement\, and business interact ions. The ability for Speech Technology to extract layers of knowledge and assess engagement content represents the next generation of advanced spee ch solutions. Today\, the emergence of BIG DATA\, Machine Learning\, as we ll as voice enabled speech systems have required the need for effective vo ice capture and automatic speech/speaker recognition. The ability to emplo y speech and language technology to assess human-to-human interactions off ers new research paradigms having profound impact on assessing human inter action. In this talk\, we will focus on big data naturalistic audio proces sing relating to (i) child learning spaces\, and (ii) the NASA APOLLO luna r missions. ML based technology advancements include automatic audio diari zation\, speech recognition\, and speaker recognition. Child-Teacher based assessment of conversational interactions are explored\, including keywor d and “WH-word” (e.g.\, who\, what\, etc.). Diarization processing solutio ns are applied to both classroom/learning space child speech\, as well as massive APOLLO data. CRSS-UTDallas is expanding our original Apollo-11 cor pus\, resulting in a massive multi-track audio processing challenge to mak e available 150\,000hrs of Apollo mission data to be shared with science c ommunities: (i) speech/language technology\, (ii) STEM/science and team-ba sed researchers\, and (iii) education/historical/archiving specialists. Ou r goals here are to provide resources which allow to better understand how people work/learn collaboratively together. For Apollo\, to accomplish on e of mankind’s greatest scientific/technological challenges in the last ce ntury.
\nBiography
\nJohn H.L. Hansen\, recei ved Ph.D. & M.S. degrees from Georgia Institute of Technology\, and B.S.E. E. from Rutgers Univ. He joined Univ. of Texas at Dallas (UTDallas) in 200 5\, where he currently serves as Associate Dean for Research\, Prof. of EC E\, Distinguished Univ. Chair in Telecom. Engineering\, and directs Center for Robust Speech Systems (CRSS). He is an ISCA Fellow\, IEEE Fellow\, an d has served as Member and TC-Chair of IEEE Signal Proc. Society\, Speech & Language Proc. Tech. Comm.(SLTC)\, and Technical Advisor to U.S. Delegat e for NATO (IST/TG-01). He served as ISCA President (2017-21)\, continues to serve on ISCA Board (2015-23) as Treasurer\, has supervised 99 PhD/MS t hesis candidates (EE\,CE\,BME\,TE\,CS\,Ling.\,Cog.Sci.\,Spch.Sci.\,Hear.Sc i)\, was recipient of 2020 UT-Dallas Provost’s Award for Grad. PhD Researc h Mentoring\; author/co-author of 865 journal/conference papers including 14 textbooks in the field of speech/language/hearing processing & technolo gy including coauthor of textbook Discrete-Time Processing of Speech Signa ls\, (IEEE Press\, 2000)\, and lead author of the report “The Impact of Sp eech Under ‘Stress’ on Military Speech Technology\,” (NATO RTO-TR-10\, 200 0). He served as Organizer\, Chair/Co-Chair/Tech.Chair for ISCA INTERSPEEC H-2022\, IEEE ICASSP-2010\, IEEE SLT-2014\, ISCA INTERSPEECH-2002\, and Te ch. Chair for IEEE ICASSP-2024. He received the 2022 IEEE Signal Processin g Society Leo Beranek MERITORIOUS SERVICE Award.
\n\n X-TAGS;LANGUAGE=en-US:2023\,Hansen\,March END:VEVENT BEGIN:VEVENT UID:ai1ec-23439@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nAs data-based technologies proliferate\, it is increa singly important for researchers to be aware of their work’s wider impact. Concerns like navigating the IRB and figuring out copyright and licensing issues are still key\, but the current focus shift to matters like inclus ivity\, fairness\, and transparency and their impact on the research/devel opment life cycle have added complexity to the research task. In this talk \, we will take a broad look at the various ways ethics intersects with na tural language processing\, machine learning\, and artificial intelligence research and discuss strategies and resources for managing these concerns within the broader research framework.\nBiography\nDenise is responsible for the overall operation of LDC’s External Relations group which includes intellectual property management\, licensing\, regulatory matters\, publi cations\, membership and communications. Before joining LDC\, she practice d law for over 20 years in the areas of international trade\, intellectual property and commercial litigation. She has an A.B. in Political Science from Bryn Mawr College and a Juris Doctor degree from the University of Mi ami School of Law. DTSTART;TZID=America/New_York:20230310T120000 DTEND;TZID=America/New_York:20230310T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street SEQUENCE:0 SUMMARY:Denise DiPersio (Linguistic Data Consortium\, University of Pennsyl vania) “Data and Ethics: Where Does the Twain Meet?” URL:https://www.clsp.jhu.edu/events/denise-dipersio-linguistic-data-consort ium-university-of-pennsylvania-data-and-ethics-where-does-the-twain-meet/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\n
Abstr act
\nAs data-based technologies proliferate\, it is increa singly important for researchers to be aware of their work’s wider impact. Concerns like navigating the IRB and figuring out copyright and licensing issues are still key\, but the current focus shift to matters like inclus ivity\, fairness\, and transparency and their impact on the research/devel opment life cycle have added complexity to the research task. In this talk \, we will take a broad look at the various ways ethics intersects with na tural language processing\, machine learning\, and artificial intelligence research and discuss strategies and resources for managing these concerns within the broader research framework.
\nBiography
\nDenise is responsible for the overall operation of LDC’s External Relations group which includes intellectual property management\, licensi ng\, regulatory matters\, publications\, membership and communications. Be fore joining LDC\, she practiced law for over 20 years in the areas of int ernational trade\, intellectual property and commercial litigation. She ha s an A.B. in Political Science from Bryn Mawr College and a Juris Doctor d egree from the University of Miami School of Law.
\n X-TAGS;LANGUAGE=en-US:2023\,DiPersio\,March END:VEVENT BEGIN:VEVENT UID:ai1ec-23312@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nAdvanced neural language models have grown ever large r and more complex\, pushing forward the limits of language understanding and generation\, while diminishing interpretability. The black-box nature of deep neural networks blocks humans from understanding them\, as well as trusting and using them in real-world applications. This talk will introd uce interpretation techniques that bridge the gap between humans and model s for developing trustworthy natural language processing(NLP). I will firs t show how to explain black-box models and evaluate their explanations for understanding their prediction behavior. Then I will introduce how to imp rove the interpretability of neural language models by making their decisi on-making transparent and rationalized. Finally\, I will discuss how to di agnose and improve models (e.g.\, robustness) through the lens of explanat ions. I will conclude with future research directions that are centered ar ound model interpretability and committed to facilitating communications a nd interactions between intelligent machines\, system developers\, and end users for long-term trustworthy AI.\nBiography\nHanjie Chen is a Ph.D. ca ndidate in Computer Science at the University of Virginia\, advised by Pro f. Yangfeng Ji. Her research interests lie in Trustworthy AI\, Natural Lan guage Processing (NLP)\, andInterpretable Machine Learning. She develops i nterpretation techniques to explain neural language models and make their prediction behavior transparent and reliable. She is a recipient of the Ca rlos and Esther Farrar Fellowship and the Best Poster Award at the ACM CAP WIC 2021. Her work has been published at top-tier NLP/AI conferences (e.g. \, ACL\, AAAI\, EMNLP\, NAACL) and selected by the National Center for Wom en & Information Technology (NCWIT) Collegiate Award Finalist 2021. She (a s the primary instructor) co-designed and taught the course\, Interpretabl e Machine Learning\, and was awarded the UVA CS Outstanding Graduate Teach ing Award and University-wide Graduate Teaching Awards Nominee (top 5% of graduate instructors). More details can be found athttps://www.cs.virginia .edu/~hc9mx DTSTART;TZID=America/New_York:20230313T120000 DTEND;TZID=America/New_York:20230313T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Hanjie Chen (University of Virginia) “Bridging Humans and Machines: Techniques for Trustworthy NLP” URL:https://www.clsp.jhu.edu/events/hanjie-chen-university-of-virginia/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nAdvanced neural language models have grown ever large
r and more complex\, pushing forward the limits of language understanding
and generation\, while diminishing interpretability. The black-box nature
of deep neural networks blocks humans from understanding them\, as well as
trusting and using them in real-world applications. This talk will introd
uce interpretation techniques that bridge the gap between humans and model
s for developing trustworthy natural language processing
(NLP). I will first show how to explain black-box models and evalua
te their explanations for understanding their prediction behavior. Then I
will introduce how to improve the interpretability of neural language mode
ls by making their decision-making transparent and rationalized. Finally\,
I will discuss how to diagnose and improve models (e.g.\, robustness) thr
ough the lens of explanations. I will conclude with future research direct
ions that are centered around model interpretability and committed to faci
litating communications and interactions between intelligent machines\, sy
stem developers\, and end users for long-term trustworthy AI.
Hanjie Chen is a Ph.D. candidate in Compute r Science at the University of Virginia\, advised by Prof. Yangfeng Ji. He r research interests lie in Trustworthy AI\, Natural Language Processing ( NLP)\, and
\n X-TAGS;LANGUAGE=en-US:2023\,Chen\,February END:VEVENT BEGIN:VEVENT UID:ai1ec-23505@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nRecent advances in large pretrained language models h ave unlocked new exciting applications for Natural Language Generation for creative tasks\, such as lyrics or humour generation. In this talk we wil l discuss recent works by our team at Alexa AI and discuss current challen ges: (1) Pun understanding and generation: We release new datasets for pun understanding and the novel task of context-situated pun generation\, and demonstrate the value of our annotations for pun classification and gener ation tasks. (2) Song lyric generation: we design a hierarchical lyric gen eration framework that enables us to generate pleasantly-singable lyrics w ithout training on melody-lyric aligned data\, and show that our approach is competitive with strong baselines supervised on parallel data. (3) Crea te with Alexa: a multimodal story creation experience recently launched on Alexa devices\, which leverages story text generation models in tandem wi th story visualization and background music generation models to produce m ultimodal stories for kids.\nBiography\nAlessandra Cervone is an Applied S cientist in the Natural Understanding team at Amazon Alexa AI. Alessandra holds an MSc in Speech and Language Processing from University of Edinburg h and a PhD in CS from University of Trento (Italy). During her PhD\, Ales sandra worked on computational models of coherence in open-domain dialogue advised by Giuseppe Riccardi. In the first year of the PhD\, she was the team leader of one of the teams selected to compete in the first edition o f the Alexa Prize. More recently\, her research interests have been focuse d on natural language generation and its evaluation\, in particular in the context of creative AI applications. DTSTART;TZID=America/New_York:20230317T120000 DTEND;TZID=America/New_York:20230317T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Alessandra Cervone (Amazon) “Controllable Text Generation for Creat ive Applications URL:https://www.clsp.jhu.edu/events/alexxandra-cervone-amazon/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\n Interpretable Machine Learning. She dev elops interpretation techniques to explain neural language models and make their prediction behavior transparent and reliable. She is a recipient of the Carlos and Esther Farrar Fellowship and the Best Poster Award at the ACM CAPWIC 2021. Her work has been published at top-tier NLP/AI conference s (e.g.\, ACL\, AAAI\, EMNLP\, NAACL) and selected by the National Center for Women & Information Technology (NCWIT) Collegiate Award Finalist 2021. She (as the primary instructor) co-designed and taught the course\, Inter pretable Machine Learning\, and was awarded the UVA CS Outstanding Graduat e Teaching Award and University-wide Graduate Teaching Awards Nominee (top 5% of graduate instructors). More details can be found at https://www.cs.virginia.edu/~hc9mxAbstr act
\nRecent advances in large pretrain ed language models have unlocked new exciting applications for Natural Lan guage Generation for creative tasks\, such as lyrics or humour generation. In this talk we will discuss recent works by our team at Alexa AI and dis cuss current challenges: (1) Pun understanding and generation: We release new datasets for pun understanding and the novel task of context-situated pun generation\, and demonstrate the value of our annotations for pun clas sification and generation tasks. (2) Song lyric generation: we design a hi erarchical lyric generation framework that enables us to generate pleasant ly-singable lyrics without training on melody-lyric aligned data\, and sho w that our approach is competitive with strong baselines supervised on par allel data. (3) Create with Alexa: a multimodal story creation experience recently launched on Alexa devices\, which leverages story text generation models in tandem with story visualization and background music generation models to produce multimodal stories for kids.
\nBiography< /strong>
\nAlessandra Cervone is an Applied Scientist in the Natural Understanding team at Amazon Alexa AI. Alessandra holds an MSc in Speech and Language Processing from University of Edinburgh and a PhD in CS from University of Trento (Italy). During her PhD\, Alessandra worked on comput ational models of coherence in open-domain dialogue advised by Giuseppe Ri ccardi. In the first year of the PhD\, she was the team leader of one of t he teams selected to compete in the first edition of the Alexa Prize. More recently\, her research interests have been focused on natural language g eneration and its evaluation\, in particular in the context of creative AI applications.
\n \\nAbstr act
\nDespite many recent advances in automatic speech reco gnition (ASR)\, linguists and language communities engaged in language doc umentation projects continue to face the obstacle of the “transcription bo ttleneck”. Researchers in NLP typically do not distinguish between widely spoken languages that currently happen to have few training resources and endangered languages that will never have abundant data. As a result\, we often fail to thoroughly explore when ASR is helpful for language document ation\, what architectures work best for the sorts of languages that are i n need of documentation\, and how data can be collected and organized to p roduce optimal results. In this talk I describe several projects that atte mpt to bridge the gap between the promise of ASR for language documentatio n and the reality of using this technology in real-world settings.
\nBiography
\nAbstr act
\nHow important are different temporal speech modulations for speec h recognition? We answer this question from two complementary perspectives . Firstly\, we quantify the amount of phonetic information in the modulati on spectrum of speech by computing the mutual information between temporal modulations with frame-wise phoneme labels. Looking from another perspect ive\, we ask – which speech modulations an Automatic Speech Recognition (A SR) system prefers for its operation. Data-driven weights are learned over the modulation spectrum and optimized for an end-to-end ASR task. Both me thods unanimously agree that speech information is mostly contained in slo w modulation. Maximum mutual information occurs around 3-6 Hz which also h appens to be the range of modulations most preferred by the ASR. In additi on\, we show that the incorporation of this knowledge into ASRs significan tly reduces their dependency on the amount of training data.
\n< p> \nLearning How to Play With The Machines: Taking Stock of Where the Collaboration Between Computational and Social Science Stands
\n\n
Speakers: Jeff Gill\, Ernesto Calvo\, Hale Sirin and Antonios Anastasopoulos
\n X-TAGS;LANGUAGE=en-US:2023\,April\,APSA Roundtable END:VEVENT BEGIN:VEVENT UID:ai1ec-23586@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION: DTSTART;TZID=America/New_York:20230410T120000 DTEND;TZID=America/New_York:20230410T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Student Seminar – Ruizhe Huang URL:https://www.clsp.jhu.edu/events/student-seminar-ruizhe-huang/ X-COST-TYPE:free X-TAGS;LANGUAGE=en-US:2023\,April\,Huang END:VEVENT BEGIN:VEVENT UID:ai1ec-23588@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nAdvances in open domain Large Language Models (LLMs) starting with BERT and more recently with GPT-4\, PaLM\, and LLaMA have fa cilitated dramatic improvements in conversational systems. These improveme nts include an unprecedented breadth of conversational interactions betwee n humans and machines while maintaining and sometimes surpassing the accur acy of systems trained specifically for known\, closed domains. However\, many applications still require higher levels of accuracy than pre-trained LLMs can provide. There are many studies underway to accomplish this. Bro adly speaking\, the methods assume the pre-trained models are fixed (due t o cost/time)\, and instead look to various augmentation methods including prompting strategies and model adaptation/fine-tuning.\nOne augmentation s trategy leverages the context of the conversation. For example\, who are t he participants and what is known about these individuals (personal contex t)\, what was just said (dialogue context)\, where is the conversation tak ing place (geo context)\, what time of day and season is it (time context) \, etc. A powerful form of context is the shared visual setting of the co nversation between the human(s) and machine. The shared visual scene may b e from a device (phone\, smart glasses) or represented on a screen (browse r\, maps\, etc.) The elements in the visual context can be exploited by gr ounding the natural language conversational interaction\, thereby changing the priors of certain concepts and increasing the accuracy of the system. In this talk\, I will present some of my historical work in this area as well as my recent work in the AI Virtual Assistant (AVA) Lab at Georgia Te ch.\nBio\nDr. Larry Heck is a Professor with a joint appointment in the Sc hool of Electrical and Computer Engineering and the School of Interactive Computing at the Georgia Institute of Technology. He holds the Rhesa S. Fa rmer Distinguished Chair of Advanced Computing Concepts and is a Georgia R esearch Alliance Eminent Scholar. His received the BSEE from Texas Tech Un iversity (1986)\, and MSEE and PhD EE from the Georgia Institute of Techno logy (1989\,1991). He is a Fellow of the IEEE\, inducted into the Academy of Distinguished Engineering Alumni at Georgia Tech and received the Disti nguished Engineer Award from the Texas Tech University Whitacre College of Engineering. He was a Senior Research Engineer with SRI (1992-98)\, Vice President of R&D at Nuance (1998-2005)\, Vice President of Search and Adve rtising Sciences at Yahoo! (2005-2009)\, Chief Scientist of the Microsoft Speech products and Distinguished Engineer in Microsoft Research (2009-201 4)\, Principal Scientist with Google Research (2014-2017)\, and CEO of Viv Labs and SVP at Samsung (2017-2021).\n\n DTSTART;TZID=America/New_York:20230414T120000 DTEND;TZID=America/New_York:20230414T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Larry Heck (Georgia Institute of Technology) “The AVA Digital Human : Improving Conversational Interactions through Visually Situated Context” URL:https://www.clsp.jhu.edu/events/larry-heck-georgia-institute-of-technol ogy-the-ava-digital-human-improving-conversational-interactions-through-vi sually-situated-context/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nAdvances in open domain Large Lan guage Models (LLMs) starting with BERT and more recently with GPT-4\, PaLM \, and LLaMA have facilitated dramatic improvements in conversational syst ems. These improvements include an unprecedented breadth of conversational interactions between humans and machines while maintaining and sometimes surpassing the accuracy of systems trained specifically for known\, closed domains. However\, many applications still require higher levels of accur acy than pre-trained LLMs can provide. There are many studies underway to accomplish this. Broadly speaking\, the methods assume the pre-trained mod els are fixed (due to cost/time)\, and instead look to various augmentatio n methods including prompting strategies and model adaptation/fine-tuning.
\nOne augmentation strategy leverages the conte xt of the conversation. For example\, who are the participants and what is known about these individuals (personal context)\, what was just said (di alogue context)\, where is the conversation taking place (geo context)\, w hat time of day and season is it (time context)\, etc. A powerful form of context is the shared visual setting of the conversation between the huma n(s) and machine. The shared visual scene may be from a device (phone\, sm art glasses) or represented on a screen (browser\, maps\, etc.) The elemen ts in the visual context can be exploited by grounding the natural languag e conversational interaction\, thereby changing the priors of certain conc epts and increasing the accuracy of the system. In this talk\, I will pres ent some of my historical work in this area as well as my recent work in t he AI Virtual Assistant (AVA) Lab at Georgia Tech.
\nBio
\nDr. Larry Heck is a Professor with a joi nt appointment in the School of Electrical and Computer Engineering and th e School of Interactive Computing at the Georgia Institute of Technology. He holds the Rhesa S. Farmer Distinguished Chair of Advanced Computing Con cepts and is a Georgia Research Alliance Eminent Scholar. His received the BSEE from Texas Tech University (1986)\, and MSEE and PhD EE from the Geo rgia Institute of Technology (1989\,1991). He is a Fellow of the IEEE\, in ducted into the Academy of Distinguished Engineering Alumni at Georgia Tec h and received the Distinguished Engineer Award from the Texas Tech Univer sity Whitacre College of Engineering. He was a Senior Research Engineer wi th SRI (1992-98)\, Vice President of R&D at Nuance (1998-2005)\, Vice Pres ident of Search and Advertising Sciences at Yahoo! (2005-2009)\, Chief Sci entist of the Microsoft Speech products and Distinguished Engineer in Micr osoft Research (2009-2014)\, Principal Scientist with Google Research (201 4-2017)\, and CEO of Viv Labs and SVP at Samsung (2017-2021).
\n\n
\n X-TAGS;LANGUAGE=en-US:2023\,April\,Heck END:VEVENT BEGIN:VEVENT UID:ai1ec-23590@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nMachine Translation has the ultimate goal of eliminat ing language barriers. However\, the area has focused mainly on a few lang uages\, leaving many low-resource languages without support. In this talk\ , I will discuss the challenges of bringing translation support for 200 wr itten languages and beyond.\n\nFirst\, I talk about the No Language Left B ehind Project\, where we took on this challenge by first contextualizing t he need for low-resource language translation support through exploratory interviews with native speakers. Then\, we created datasets and models aim ed at narrowing the performance gap between low and high-resource language s. We proposed multiple architectural and training improvements to counter act over-fitting while training on thousands of language-pairs/tasks. We e valuated the performance of over 40\,000 different translation directions. \n\nAfterwards\, I’ll discuss the challenges of pushing translation perfor mance beyond text for languages that don’t have written standards like Hok kien.\nOur models achieve state-of-the-art performance and lay important g roundwork towards realizing a universal translation system. At the same ti me\, we keep making open-source contributions for everyone to keep advanci ng the research for the languages they care about.\nBio\nPaco is Research Scientist Manager supporting translation teams in Meta AI (FAIR). He works in the field of machine translation with a focus on low-resource translat ion (e.g. NLLB\, FLORES) and the aim to break language barriers. He joined Meta in 2016. His research has been published in top-tier NLP venues like ACL\, EMNLP. He was the co-chair of the Research director at AMTA (2020-2 022). He has ave organized several research competitions focused on low-re source translation and data filtering. Paco obtained his PhD from the ITES M in Mexico\, was a visiting scholar at the LTI-CMU from 2008-2009\, and p articipated in DARPA’s GALE evaluation program. Paco was a post-doc and sc ientist at Qatar Computing Research Institute in Qatar in 2012-2016 DTSTART;TZID=America/New_York:20230417T120000 DTEND;TZID=America/New_York:20230417T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Paco Guzman (Meta AI) “Building a Universal Translation System to B reak Down Language Barriers” URL:https://www.clsp.jhu.edu/events/paco-guzman-meta-ai/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n
\\nAbstr act
\nOur models achieve state-of-the-art performance and lay important groundwork towards realizing a universal translation system. At the same time\, we keep maki ng open-source contributions for everyone to keep advancing the research f or the languages they care about.
\nBio
\nPac o is Research Scientist Manager supporting translation teams in Meta AI (F AIR). He works in the field of machine translation with a focus on low-res ource translation (e.g. NLLB\, FLORES) and the aim to break language barri ers. He joined Meta in 2016. His research has been published in top-tier N LP venues like ACL\, EMNLP. He was the co-chair of the Research director a t AMTA (2020-2022). He has ave organized several research competitions foc used on low-resource translation and data filtering. Paco obtained his PhD from the ITESM in Mexico\, was a visiting scholar at the LTI-CMU from 200 8-2009\, and participated in DARPA’s GALE evaluation program. Paco was a p ost-doc and scientist at Qatar Computing Research Institute in Qatar in 20 12-2016
\n X-TAGS;LANGUAGE=en-US:2023\,April\,Guzman END:VEVENT BEGIN:VEVENT UID:ai1ec-23592@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nLarge language models (LLMs) have ushered in exciting capabilities in language understanding and text generation\, with systems like ChatGPT holding fluent dialogs with users and being almost indisting uishable from humans. While this has obviously raised conversational syste ms and chatbots to a new level\, it also presents exciting new opportuniti es for building artificial agents with improved decision making capabiliti es. Specifically\, the ability to reason with language can allow us to bui ld agents that can 1) execute complex action sequences to effect change in the world\, 2) learn new skills by ‘reading’ in addition to ‘doing’\, and 3) allow for easier personalization and control over their behavior. In t his talk\, I will demonstrate how we can build such language-enabled agent s that exhibit the above traits across various use cases such as multi-hop question answering\, web interaction\, and robotic tool manipulation. In the end\, I will also discuss some dangers of using these LLM-based system s and some challenges that lie ahead in ensuring their safe use.\nBiograph y\nKarthik Narasimhan is an assistant professor in the Computer Science de partment at Princeton University and a co-Director of the Princeton NLP gr oup. His research spans the areas of natural language processing and reinf orcement learning\, with the goal of building intelligent agents that lear n to operate in the world through both their own experience (”doing things ”) and leveraging existing human knowledge (”reading about things”). Karth ik received his PhD from MIT in 2017\, and spent a year as a visiting rese arch scientist at OpenAI contributing to the GPT language model\, prior to joining Princeton in 2018. His research has been recognized by the NSF CA REER\, a Google Research Scholar Award\, an Amazon research award (2019)\, Bell Labs runner-up prize and outstanding paper awards at EMNLP (2015\, 2 016) and NeurIPS (2022). DTSTART;TZID=America/New_York:20230421T120000 DTEND;TZID=America/New_York:20230421T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Karthik Narasimhan (Princeton University) ” Towards General-Purpose Language-Enabled Agents: Machines that can Read\, Think and Act” URL:https://www.clsp.jhu.edu/events/karthik-narasimhan-princeton-university / X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nLarge language models (LLMs) have ushered in exciting capabilities in language understanding and text generation\, with systems like ChatGPT holding fluent dialogs with users and being almost indisting uishable from humans. While this has obviously raised conversational syste ms and chatbots to a new level\, it also presents exciting new opportuniti es for building artificial agents with improved decision making capabiliti es. Specifically\, the ability to reason with language can allow us to bui ld agents that can 1) execute complex action sequences to effect change in the world\, 2) learn new skills by ‘reading’ in addition to ‘doing’\, and 3) allow for easier personalization and control over their behavior. In t his talk\, I will demonstrate how we can build such language-enabled agent s that exhibit the above traits across various use cases such as multi-hop question answering\, web interaction\, and robotic tool manipulation. In the end\, I will also discuss some dangers of using these LLM-based system s and some challenges that lie ahead in ensuring their safe use.
\n< strong>Biography
\nKarthik Narasimhan is an assistan t professor in the Computer Science department at Princeton University and a co-Director of the Princeton NLP group. His research spans the areas of natural language processing and reinforcement learning\, with the goal of building intelligent agents that learn to operate in the world through bo th their own experience (”doing things”) and leveraging existing human kno wledge (”reading about things”). Karthik received his PhD from MIT in 2017 \, and spent a year as a visiting research scientist at OpenAI contributin g to the GPT language model\, prior to joining Princeton in 2018. His rese arch has been recognized by the NSF CAREER\, a Google Research Scholar Awa rd\, an Amazon research award (2019)\, Bell Labs runner-up prize and outst anding paper awards at EMNLP (2015\, 2016) and NeurIPS (2022).
\n X-TAGS;LANGUAGE=en-US:2023\,April\,Narasimhan END:VEVENT BEGIN:VEVENT UID:ai1ec-23606@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION: DTSTART;TZID=America/New_York:20230424T120000 DTEND;TZID=America/New_York:20230424T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Student Seminar – Brian Lu URL:https://www.clsp.jhu.edu/events/student-seminar-brian-lu/ X-COST-TYPE:free X-TAGS;LANGUAGE=en-US:2023\,April\,Lu END:VEVENT BEGIN:VEVENT UID:ai1ec-23608@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nAutomated analysis of student writing has the potenti al to provide alternatives to selected-response questions such as multiple choice\, and to enable teachers and instructors to assess students’ reaso ning skills based on their long-form writing. Further\, automated support to assess both short answers and long passages could provide students with a smoother trajectory towards mastery of written communication. Our meth ods focus on the specific ideas students express to support formative asse ssment through different kinds of feedback\, which aims to scaffold their abilities to reason and communicate. In this talk I review our work in the PSU NLP lab on methods for automated assessment of different forms of stu dent writing\, from younger and older students. I will briefly illustrate highly curated datasets created in collaboration with researchers in STEM education\, results from deployment of an older content analysis tool on middle school physics essays\, and very preliminary results on assessment of college students’ physics lab reports. I will also present our current work on short answer assessment using a novel recurrent relation network that incorporates contrastive learning.\nBio\nBecky Passonneau has been a Professor in the Department of Computer Science and Engineering at Penn St ate University since 2016\, when she joined as the first NLP researcher. S ince that time the NLP faculty has grown to include Rui Zhang and Wenpeng Yin. Becky’s research in natural language processing addresses computation al pragmatics\, meaning the investigation of language as a system of inter active behavior that serves a wide range of purposes. She received her PhD in Linguistics from the University of Chicago in 1985\, and worked at sev eral academic and industry research labs before joining Penn State. Her wo rk is reported in over 140 publications in journals and refereed conferenc e proceedings\, and has been funded through 27 sponsored projects from 16 sources\, including government agencies\, corporate sponsors\, corporate gifts\, and foundations.. DTSTART;TZID=America/New_York:20230428T120000 DTEND;TZID=America/New_York:20230428T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Becky Passonneau (Penn State University) ” Automated Support to Sca ffold Students’ Short- and Long-form STEM Writing” URL:https://www.clsp.jhu.edu/events/becky-passonneau-penn-state-university- automated-support-to-scaffold-students-short-and-long-form-stem-writing/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nAutomated analysis of student writing has the potenti al to provide alternatives to selected-response questions such as multiple choice\, and to enable teachers and instructors to assess students’ reaso ning skills based on their long-form writing. Further\, automated support to assess both short answers and long passages could provide students with a smoother trajectory towards mastery of written communication. Our meth ods focus on the specific ideas students express to support formative asse ssment through different kinds of feedback\, which aims to scaffold their abilities to reason and communicate. In this talk I review our work in the PSU NLP lab on methods for automated assessment of different forms of stu dent writing\, from younger and older students. I will briefly illustrate highly curated datasets created in collaboration with researchers in STEM education\, results from deployment of an older content analysis tool on middle school physics essays\, and very preliminary results on assessment of college students’ physics lab reports. I will also present our current work on short answer assessment using a novel recurrent relation network that incorporates contrastive learning.
\nBio
\nBecky Passonneau has been a Professor in the Department of Computer Sci ence and Engineering at Penn State University since 2016\, when she joined as the first NLP researcher. Since that time the NLP faculty has grown to include Rui Zhang and Wenpeng Yin. Becky’s research in natural language p rocessing addresses computational pragmatics\, meaning the investigation o f language as a system of interactive behavior that serves a wide range of purposes. She received her PhD in Linguistics from the University of Chic ago in 1985\, and worked at several academic and industry research labs be fore joining Penn State. Her work is reported in over 140 publications in journals and refereed conference proceedings\, and has been funded through 27 sponsored projects from 16 sources\, including government agencies\, corporate sponsors\, corporate gifts\, and foundations..
\n X-TAGS;LANGUAGE=en-US:2023\,April\,Passonneau END:VEVENT BEGIN:VEVENT UID:ai1ec-23880@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION: DTSTART;TZID=America/New_York:20230828T120000 DTEND;TZID=America/New_York:20230828T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street SEQUENCE:0 SUMMARY:CLSP Town Hall – Welcome New Students\, Introductions and CLSP Over view URL:https://www.clsp.jhu.edu/events/clsp-town-hall-welcome-new-students-int roductions-and-clsp-overview/ X-COST-TYPE:free X-TAGS;LANGUAGE=en-US:2023\,August\,Town Hall END:VEVENT BEGIN:VEVENT UID:ai1ec-23882@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nLarge language models (LLMs) have demonstrated incred ible power\, but they also possess vulnerabilities that can lead to misuse and potential attacks. In this presentation\, we will address two fundame ntal questions regarding the responsible utilization of LLMs: (1) How can we accurately identify AI-generated text? (2) What measures can safeguard the intellectual property of LLMs? We will introduce two recent watermarki ng techniques designed for text and models\, respectively. Our discussion will encompass the theoretical underpinnings that ensure the correctness o f watermark detection\, along with robustness against evasion attacks. Fur thermore\, we will showcase empirical evidence validating their effectiven ess. These findings establish a solid technical groundwork for policymaker s\, legal professionals\, and generative AI practitioners alike.\nBiograph y\nLei Li is an Assistant Professor in Language Technology Institute at Ca rnegie Mellon University. He received Ph.D. from Carnegie Mellon Universit y School of Computer Science. He is a recipient of ACL 2021 Best Paper Awa rd\, CCF Young Elite Award in 2019\, CCF distinguished speaker in 2017\, W u Wen-tsün AI prize in 2017\, and 2012 ACM SIGKDD dissertation award (runn er-up)\, and is recognized as Notable Area Chair of ICLR 2023. Previously\ , he was a faculty member at UC Santa Barbara. Prior to that\, he founded ByteDance AI Lab in 2016 and led its research in NLP\, ML\, Robotics\, an d Drug Discovery. He launched ByteDance’s machine translation system VolcT rans and AI writing system Xiaomingbot\, serving one billion users. DTSTART;TZID=America/New_York:20230901T120000 DTEND;TZID=America/New_York:20230901T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Lei Li (Carnegie Mellon University) “Empowering Responsible Use of Large Language Models” URL:https://www.clsp.jhu.edu/events/lei-li-carnegie-mellon-university-empow ering-responsible-use-of-large-language-models/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nLarge language models (LLMs) have demonstrated incred ible power\, but they also possess vulnerabilities that can lead to misuse and potential attacks. In this presentation\, we will address two fundame ntal questions regarding the responsible utilization of LLMs: (1) How can we accurately identify AI-generated text? (2) What measures can safeguard the intellectual property of LLMs? We will introduce two recent watermarki ng techniques designed for text and models\, respectively. Our discussion will encompass the theoretical underpinnings that ensure the correctness o f watermark detection\, along with robustness against evasion attacks. Fur thermore\, we will showcase empirical evidence validating their effectiven ess. These findings establish a solid technical groundwork for policymaker s\, legal professionals\, and generative AI practitioners alike.
\n< strong>Biography
\nLei Li is an Assistant Professor in Lang uage Technology Institute at Carnegie Mellon University. He received Ph.D. from Carnegie Mellon University School of Computer Science. He is a recip ient of ACL 2021 Best Paper Award\, CCF Young Elite Award in 2019\, CCF di stinguished speaker in 2017\, Wu Wen-tsün AI prize in 2017\, and 2012 ACM SIGKDD dissertation award (runner-up)\, and is recognized as Notable Area Chair of ICLR 2023. Previously\, he was a faculty member at UC Santa Barba ra. Prior to that\, he founded ByteDance AI Lab in 2016 and led its resea rch in NLP\, ML\, Robotics\, and Drug Discovery. He launched ByteDance’s m achine translation system VolcTrans and AI writing system Xiaomingbot\, se rving one billion users.
\n X-TAGS;LANGUAGE=en-US:2023\,Li\,September END:VEVENT BEGIN:VEVENT UID:ai1ec-23886@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nThe arms race to build increasingly larger\, powerful language models (LMs) in the past year has been remarkable. Yet incorpora ting LMs effectively into practical applications that facilitate manual wo rkflows remains challenging. I will discuss LMs’ limiting factors and our efforts to overcome them. I will start with challenges surrounding efficie nt and robust LM alignment. I will share insights from our recent paper “S elf-Instruct” (ACL 2023)\, where we used vanilla (unaligned) LMs for align ing itself\, an approach that has yielded some success. Then\, I will move on to the challenge of tracing the output of LMs to reliable sources\, a weakness that makes them prone to hallucinations. I will discuss our recen t approach of ‘according-to’ prompting\, which steers LMs to quote directl y from sources observed in its pre-training. If time permits\, I will disc uss our ongoing project to adapt LMs to interact with web pages. Throughou t the presentation\, I will highlight our progress\, and end with question s about our future progress.\nBiography\nDaniel Khashabi is an assistant p rofessor in computer science at Johns Hopkins University and the Center fo r Language and Speech Processing (CLSP) member. He is interested in buildi ng reasoning-driven modular NLP systems that are robust\, transparent\, an d communicative\, particularly those that use natural language as the comm unication medium. Khashabi has published over 40 papers on natural languag e processing and AI in top-tier venues. His work touches upon developing. His research has won the ACL 2023 Outstanding Paper Award\, NAACL 2022 Bes t Paper Award\, research gifts from the Allen Institute for AI\, and an Am azon Research Award 2023. Before joining Hopkins\, he was a postdoctoral f ellow at the Allen Institute for AI (2019-2022) and obtained a Ph.D. from the University of Pennsylvania in 2019. DTSTART;TZID=America/New_York:20230908T120000 DTEND;TZID=America/New_York:20230908T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Daniel Khashabi (Johns Hopkins University) “Building More Helpful L anguage Models” URL:https://www.clsp.jhu.edu/events/daniel-khashabi-johns-hopkins-universit y/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nThe arms race to build increasingly larger\, powerful language models (LMs) in the past year has been remarkable. Yet incorpora ting LMs effectively into practical applications that facilitate manual wo rkflows remains challenging. I will discuss LMs’ limiting factors and our efforts to overcome them. I will start with challenges surrounding efficie nt and robust LM alignment. I will share insights from our recent paper “Self-Instruct” (ACL 2023)\, where we used vanilla (unaligned) LMs for aligning itself\, an approach that has yielded some success. Then\, I will move on to the challenge of t racing the output of LMs to reliable sources\, a weakness that makes them prone to hallucinations. I will discuss our recent approach of ‘according-to’ prompting\, which steers LM s to quote directly from sources observed in its pre-training. If time per mits\, I will discuss our ongoing project to adapt LMs to interact with we b pages. Throughout the presentation\, I will highlight our progress\, and end with questions about our future progress.
\nBiography strong>
\nDaniel Khashabi is an assistant professor in computer science at Johns Hopkins University and the Center for Language and Speech Pr ocessing (CLSP) member. He is interested in building reasoning-driven modu lar NLP systems that are robust\, transparent\, and communicative\, partic ularly those that use natural language as the communication medium. Khasha bi has published over 40 papers on natural language processing and AI in t op-tier venues. His work touches upon developing. His research has won the ACL 2023 Outstanding Paper Award\, NAACL 2022 Best Paper Award\, research gifts from the Allen Institute for AI\, and an Amazon Research Award 2023 . Before joining Hopkins\, he was a postdoctoral fellow at the Allen Insti tute for AI (2019-2022) and obtained a Ph.D. from the University of Pennsy lvania in 2019.
\n X-TAGS;LANGUAGE=en-US:2023\,Khashabi\,September END:VEVENT BEGIN:VEVENT UID:ai1ec-23888@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION:Abstract\nEmbedding text sequences is a widespread requirement in modern language understanding. Existing approaches focus largely on con stant-size representations. This is problematic\, as the amount of informa tion contained in text often varies with the length of the input. We propo se a solution called Nugget\, which encodes language into a representation based on a dynamically selected subset of input tokens. These nuggets are learned through tasks like autoencoding and machine translation\, and int uitively segment language into meaningful units. We demonstrate Nugget out performs related approaches in tasks involving semantic comparison. Finall y\, we illustrate these compact units allow for expanding the contextual w indow of a language model (LM)\, suggesting new future LMs that can condit ion on significantly larger amounts of content. DTSTART;TZID=America/New_York:20230911T120000 DTEND;TZID=America/New_York:20230911T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Student Seminar – Guanghui Qin “Nugget: Neural Agglomerative Embedd ings of Text (ICML 2023)” URL:https://www.clsp.jhu.edu/events/student-seminar-guanghui-qin/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nEmbedding text sequ ences is a widespread requirement in modern language understanding. Existi ng approaches focus largely on constant-size representations. This is prob lematic\, as the amount of information contained in text often varies with the length of the input. We propose a solution called Nugget\, which enco des language into a representation based on a dynamically selected subset of input tokens. These nuggets are learned through tasks like autoencoding and machine translation\, and intuitively segment language into meaningfu l units. We demonstrate Nugget outperforms related approaches in tasks inv olving semantic comparison. Finally\, we illustrate these compact units al low for expanding the contextual window of a language model (LM)\, suggest ing new future LMs that can condition on significantly larger amounts of c ontent.
\n X-TAGS;LANGUAGE=en-US:2023\,Qin\,September END:VEVENT BEGIN:VEVENT UID:ai1ec-23892@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nThe growing power in computing and AI promises a near -term future of human-machine teamwork. In this talk\, I will present my r esearch group’s efforts in understanding the complex dynamics of human-mac hine interaction and designing intelligent machines aimed to assist and co llaborate with people. I will focus on 1) tools for onboarding machine tea mmates and authoring machine assistance\, 2) methods for detecting\, and b roadly managing\, errors in collaboration\, and 3) building blocks of know ledge needed to enable ad hoc human-machine teamwork. I will also highligh t our recent work on designing assistive\, collaborative machines to suppo rt older adults aging in place.\nBiography\nChien-Ming Huang is the John C . Malone Assistant Professor in the Department of Computer Science at the Johns Hopkins University. His research focuses on designing interactive AI aimed to assist and collaborate with people. He publishes in top-tier ven ues in HRI\, HCI\, and robotics including Science Robotics\, HRI\, CHI\, a nd CSCW. His research has received media coverage from MIT Technology Revi ew\, Tech Insider\, and Science Nation. Huang completed his postdoctoral t raining at Yale University and received his Ph.D. in Computer Science at t he University of Wisconsin–Madison. He is a recipient of the NSF CAREER aw ard. https://www.cs.jhu.edu/~cmhuang/ DTSTART;TZID=America/New_York:20230915T120000 DTEND;TZID=America/New_York:20230915T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Chien-Ming Huang (Johns Hopkins University) “Becoming Teammates: De signing Assistive\, Collaborative Machines” URL:https://www.clsp.jhu.edu/events/chien-ming-huang-johns-hopkins-universi ty/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nThe growing power in computing and AI promises a near -term future of human-machine teamwork. In this talk\, I will present my r esearch group’s efforts in understanding the complex dynamics of human-mac hine interaction and designing intelligent machines aimed to assist and co llaborate with people. I will focus on 1) tools for onboarding machine tea mmates and authoring machine assistance\, 2) methods for detecting\, and b roadly managing\, errors in collaboration\, and 3) building blocks of know ledge needed to enable ad hoc human-machine teamwork. I will also highligh t our recent work on designing assistive\, collaborative machines to suppo rt older adults aging in place.
\nBiography
\nChien-Ming Huang is the John C. Malone Assistant Professor in the Departm ent of Computer Science at the Johns Hopkins University. His research focu ses on designing interactive AI aimed to assist and collaborate with peopl e. He publishes in top-tier venues in HRI\, HCI\, and robotics including S cience Robotics\, HRI\, CHI\, and CSCW. His research has received media co verage from MIT Technology Review\, Tech Insider\, and Science Nation. Hua ng completed his postdoctoral training at Yale University and received his Ph.D. in Computer Science at the University of Wisconsin–Madison. He is a recipient of the NSF CAREER award. https://www .cs.jhu.edu/~cmhuang/
\n X-TAGS;LANGUAGE=en-US:2023\,Huang\,September END:VEVENT BEGIN:VEVENT UID:ai1ec-23894@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nThe use of NLP in the realm of financial technology i s broad and complex\, with applications ranging from sentiment analysis an d named entity recognition to question answering. Large Language Models (L LMs) have been shown to be effective on a variety of tasks\; however\, no LLM specialized for the financial domain has been reported in the literatu re. In this work\, we present BloombergGPT\, a 50 billion parameter langua ge model that is trained on a wide range of financial data. We construct a 363 billion token dataset based on Bloomberg’s extensive data sources\, p erhaps the largest domain-specific dataset yet\, augmented with 345 billio n tokens from general-purpose datasets. We validate BloombergGPT on stand ard LLM benchmarks\, open financial benchmarks\, and a suite of internal b enchmarks that most accurately reflect our intended usage. Our mixed datas et training leads to a model that outperforms existing models on financial tasks by significant margins without sacrificing performance on general L LM benchmarks. Additionally\, we explain our modeling choices\, training p rocess\, and evaluation methodology.\nBiography\nMark Dredze is the John C Malone Professor of Computer Science at Johns Hopkins University and the Director of Research (Foundations of AI) for the JHU AI-X Foundry. He deve lops Artificial Intelligence Systems based on natural language processing and explores applications to public health and medicine.\nProf. Dredze is affiliated with the Malone Center for Engineering in Healthcare\, the Cent er for Language and Speech Processing\, among others. He holds a joint app ointment in the Biomedical Informatics & Data Science Section (BIDS)\, und er the Department of Medicine (DOM)\, Division of General Internal Medicin e (GIM) in the School of Medicine. He obtained his PhD from the University of Pennsylvania in 2009. DTSTART;TZID=America/New_York:20230918T120000 DTEND;TZID=America/New_York:20230918T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Mark Dredze (Johns Hopkins University) “BloombergGPT: A Large Langu age Model for Finance” URL:https://www.clsp.jhu.edu/events/mark-dredze-johns-hopkins-university/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nThe use of NLP in the realm of financial technology i s broad and complex\, with applications ranging from sentiment analysis an d named entity recognition to question answering. Large Language Models (L LMs) have been shown to be effective on a variety of tasks\; however\, no LLM specialized for the financial domain has been reported in the literatu re. In this work\, we present BloombergGPT\, a 50 billion parameter langua ge model that is trained on a wide range of financial data. We construct a 363 billion token dataset based on Bloomberg’s extensive data sources\, p erhaps the largest domain-specific dataset yet\, augmented with 345 billio n tokens from general-purpose datasets. We validate BloombergGPT on stand ard LLM benchmarks\, open financial benchmarks\, and a suite of internal b enchmarks that most accurately reflect our intended usage. Our mixed datas et training leads to a model that outperforms existing models on financial tasks by significant margins without sacrificing performance on general L LM benchmarks. Additionally\, we explain our modeling choices\, training p rocess\, and evaluation methodology.
\nBiography
\nMark Dredze is the John C Malone Professor of Computer Science at Jo hns Hopkins University and the Director of Research (Foundations of AI) fo r the JHU AI-X Foundry. He develops Artificial Intelligence Systems based on natural language processing and explores applications to public health and medicine.
\nProf. Dredze is affiliated with the Malone Center fo r Engineering in Healthcare\, the Center for Language and Speech Processin g\, among others. He holds a joint appointment in the Bio medical Informatics & Data Science Section (< span class='il'>BIDS)\, under the Department of Medicine (DOM)\, Di vision of General Internal Medicine (GIM) in the School of Medicine. He ob tained his PhD from the University of Pennsylvania in 2009.
\n HTML> X-TAGS;LANGUAGE=en-US:2023\,Dredze\,September END:VEVENT BEGIN:VEVENT UID:ai1ec-23983@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nVisually rich documents (scanned or digital) remain i mportant for many consumer and business use cases. During this talk we wil l share recent work from our team in the Document Intelligence Lab of Adob e Research to understand\, create\, and interact with these documents. Fi rst\, we’ll share a series of work on building models to decompose and und erstand the structure of documents to support use cases around document an alysis and accessibility. Next\, we’ll explore document semantic understan ding for a project where we convert natural language contract clauses to c ode to support business automation. Finally\, we’ll discuss DocEdit\, a mo del and dataset that enables editing structured documents from natural lan guage. \nBIOS:\nRajiv Jain is a Senior Research Scientist in the Document Intelligence Lab in Adobe Research\, where his research focuses on underst anding the layout\, content\, and interaction with documents. Prior to joi ning Adobe\, Rajiv was a consultant at DARPA\, where he worked on the Medi a Forensics Program to secure digital imagery. He previously served for 10 years as a researcher for the Department of Defense where he worked on pr ojects around large scale systems\, computer vision\, and network security . He received his PhD in computer science from the University of Maryland\ , College Park working in the field of document image analysis and retriev al.\nChris Tensmeyer primarily focuses on multi-modal document layout and content understanding as a Research Scientist in the Document Intelligence Lab of Adobe Research. Since joining Adobe 5 years ago\, his work has di rectly impacted popular Adobe features such as mobile Acrobat Liquid Mode\ , PDF table extraction\, handwriting recognition\, and scanned document de tection. Other research interests include general Computer Vision and Dee p Learning. He received his PhD in Computer Science from Brigham Young Un iversity on the topic of Deep Learning for Document Image Analysis. DTSTART;TZID=America/New_York:20230922T120000 DTEND;TZID=America/New_York:20230922T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Rajiv Jain and Chris Tensmeyer (Adobe) “Document Intelligence at Ad obe Research” URL:https://www.clsp.jhu.edu/events/rajiv-jain-and-chris-tensmeyer-adobe-do cument-intelligence-at-adobe-research/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nVisually rich document s (scanned or digital) remain important for many consumer and business use cases. During this talk we will sha re recent work from our team in the Document Intelligence Lab of Adobe Res earch to understand\, create\, and interact with these documents. First\, we’ll share a series of work on building models to decompose and understa nd the structure of documents to support use cases around document analysi s and accessibility. Next\, we’ll explore document semantic understanding for a project where we convert natural language contract clauses to code t o support business automation. Finally\, we’ll discuss DocEdit\, a model a nd dataset that enables editing structured documents from natural language .
\nBIOS:
\nRajiv Jain is a Senior Research Scientist in the Do cument Intelligence Lab in Adobe Research\, where his research focuses on understanding the layout\, content\, and interaction with documents. Prior to joining Adobe\, Rajiv was a consultant at DARPA\, where he worked on t he Media Forensics Program to secure digital imagery. He previously served for 10 years as a researcher for the Department of Defense where he worke d on projects around large scale systems\, computer vision\, and network s ecurity. He received his PhD in computer science from the University of Ma ryland\, College Park working in the field of document image analysis and retrieval.
\nChris Ten smeyer primarily focuses on multi-modal document layout and conte nt understanding as a Research Scientist in the Document Intelligence Lab of Adobe Research. Since joining Adobe 5 years ago\, his work has directl y impacted popular Adobe features such as mobile Acrobat Liquid Mode\, PDF table extraction\, handwriting recognition\, and scanned document detecti on. Other research interests include general Computer Vision and Deep Lea rning. He received his PhD in Computer Science from Brigham Young Univers ity on the topic of Deep Learning for Document Image Analysis.
\n X-TAGS;LANGUAGE=en-US:2023\,Jain and Tensmeyer\,September END:VEVENT BEGIN:VEVENT UID:ai1ec-23896@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nThe field of NLP is in the midst of a disruptive shif t\, fueled most recently by the advent of large language models (LLMs)\, w ith impacts on our methodologies\, funding and public perception. While th e core technologies and scope of real-world impact of our field may be cha nging (everything is different!)\, many of the same key challenges faced s ince the inception of our field remain (nothing has changed). In this talk I’ll describe recent work characterizing and tackling some of these chall enges\, notably: data-efficient domain adaptation and lifelong learning. I will also anchor discussion of cycles and shifts in the field by describi ng findings from a qualitative study of factors shaping the community over time\, including culture\, incentives\, and infrastructure. Through these complementary lenses into the past\, present and future\, I aim to inspir e shared hope\, excitement and discussion. \nBio\nEmma Strubell is the Raj Reddy Assistant Professor in the Language Technologies Institute in the S chool of Computer Science at Carnegie Mellon University\, and a Visiting S cientist at the Allen Institute for Artificial Intelligence. Previously sh e held research scientist roles at Google and FAIR after earning her docto ral degree in 2019 from the University of Massachusetts Amherst. Her resea rch lies at the intersection of natural language processing and machine le arning\, with a focus on providing pragmatic solutions to practitioners wh o wish to gain insights from natural language text via computation- and da ta-efficient AI. Her work has been recognized with a Madrona AI Impact Awa rd\, best paper awards at ACL and EMNLP\, and cited in news outlets includ ing the New York Times and Wall Street Journal. DTSTART;TZID=America/New_York:20230925T120000 DTEND;TZID=America/New_York:20230925T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Emma Strubell (Carnegie Mellon University) “Large Language Models: Everything’s Different and Nothing Has Changed” URL:https://www.clsp.jhu.edu/events/emma-strubell-carnegie-mellon-universit y/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nThe field of NLP i s in the midst of a disruptive shift\, fueled most recently by the advent of large language models (LLMs)\, with impacts on our methodologies\, fund ing and public perception. While the core technologies and scope of real-w orld impact of our field may be changing (everything is different!)\, many of the same key challenges faced since the inception of our field remain (nothing has changed). In this talk I’ll describe recent work characterizi ng and tackling some of these challenges\, notably: data-efficient domain adaptation and lifelong learning. I will also anchor discussion of cycles and shifts in the field by describing findings from a qualitative study of factors shaping the community over time\, including culture\, incentives\ , and infrastructure. Through these complementary lenses into the past\, p resent and future\, I aim to inspire shared hope\, excitement and discussi on.
\nBio
\n< span class='x_x_x_ContentPasted1'>Emma Strubell is the Raj Reddy Assistant Professor in the Language Technologies Institute in the School of Compute r Science at Carnegie Mellon University\, and a Visiting Scientist at the Allen Institute for Artificial Intelligence. Previously she held research scientist roles at Google and FAIR after earning her doctoral degree in 20 19 from the University of Massachusetts Amherst. Her research lies at the intersection of natural language processing and machine learning\, with a focus on providing pragmatic solutions to practitioners who wish to gain i nsights from natural language text via computation- and data-efficient AI. Her work has been recognized with a Madrona AI Impact Award\, best paper awards at ACL and EMNLP\, and cited in news outlets including the New York Times and Wall Street Journal.
\n X-TAGS;LANGUAGE=en-US:2023\,September\,Strubell END:VEVENT BEGIN:VEVENT UID:ai1ec-23898@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION:Abstract\nAny valuable NLP dataset has traditionally been shipp ed with crowdsourced categorical labels. Instructions for collecting these labels are easy to communicate and the labels themselves are easy to anno tate. However\, as self-supervision based methods are getting better at ba sically everything\, human annotations may need to provide more nuanced su pervision or enable more detailed evaluation in order to be worth further collecting. One natural extension to existing categorical annotation schem es is to obtain uncertainty information beyond a single hard label. In thi s talk\, I will discuss my recent efforts on introducing scalar labels in place of categorical labels as a form of uncertainty annotation. We demons trate that\, compared to other more obvious annotation schemes for eliciti ng uncertainty information\, scalar labels are significantly more cost-eff ective to annotate\, provide reliable evaluation\, and have a theoretical connection to existing predictive uncertainty metrics. In particular\, the y motivate using other losses as surrogates for calibration evaluation. DTSTART;TZID=America/New_York:20230929T120000 DTEND;TZID=America/New_York:20230929T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:CLSP Student Seminar – Zhengping Jiang “Scalar Labels for Capturing Human Uncertainty” URL:https://www.clsp.jhu.edu/events/clsp-student-seminar-zhengping-jiang/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nAny valuable NLP d ataset has traditionally been shipped with crowdsourced categorical labels . Instructions for collecting these labels are easy to communicate and the labels themselves are easy to annotate. However\, as self-supervision bas ed methods are getting better at basically everything\, human annotations may need to provide more nuanced supervision or enable more detailed evalu ation in order to be worth further collecting. One natural extension to ex isting categorical annotation schemes is to obtain uncertainty information beyond a single hard label. In this talk\, I will discuss my recent effor ts on introducing scalar labels in place of categorical labels as a form o f uncertainty annotation. We demonstrate that\, compared to other more obv ious annotation schemes for eliciting uncertainty information\, scalar lab els are significantly more cost-effective to annotate\, provide reliable e valuation\, and have a theoretical connection to existing predictive uncer tainty metrics. In particular\, they motivate using other losses as surrog ates for calibration evaluation.
\n X-TAGS;LANGUAGE=en-US:2023\,Jiang\,September END:VEVENT BEGIN:VEVENT UID:ai1ec-23900@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION: DTSTART;TZID=America/New_York:20231002T120000 DTEND;TZID=America/New_York:20231002T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:CLSP Student Seminar – Anna Favaro URL:https://www.clsp.jhu.edu/events/clsp-student-seminar-anna-favaro/ X-COST-TYPE:free X-TAGS;LANGUAGE=en-US:2023\,Favaro\,October END:VEVENT BEGIN:VEVENT UID:ai1ec-24115@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION:Abstract\nOur goal is to use AI to automatically find tax minim ization strategies\, an approach which we call “Shelter Check.” It would c ome in two variants. Existing-Authority Shelter Check would aim to find wh ether existing tax law authorities can be combined to create tax minimizat ion strategies\, so the IRS or Congress can shut them down. New-Authority Shelter Check would automate checking whether a new tax law authority – li ke proposed legislation or a draft court decision – would combine with exi sting authorities to create a new tax minimization strategy. We had initia lly had high hopes for GPT-* large language models for implementing Shelte r Check\, but our tests have showed that they do very poorly at basic lega l reasoning and handling legal text. So we are now creating a benchmark an d training data for LLM’s handling legal text\, hoping to spur improvement s. DTSTART;TZID=America/New_York:20231006T120000 DTEND;TZID=America/New_York:20231006T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:CLSP Student Seminar – Andrew Blair-Stanek “Shelter Check and GPT-4 ’s Bad Legal Performance” URL:https://www.clsp.jhu.edu/events/clsp-student-seminar-andrew-blair-stane k-shelter-check-and-gpt-4s-bad-legal-performance/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nOur goal is to use AI to automatically find tax minim ization strategies\, an approach which we call “Shelter Check.” It would c ome in two variants. Existing-Authority Shelter Check would aim to find wh ether existing tax law authorities can be combined to create tax minimizat ion strategies\, so the IRS or Congress can shut them down. New-Authority Shelter Check would automate checking whether a new tax law authority – li ke proposed legislation or a draft court decision – would combine with exi sting authorities to create a new tax minimization strategy. We had initia lly had high hopes for GPT-* large language models for implementing Shelte r Check\, but our tests have showed that they do very poorly at basic lega l reasoning and handling legal text. So we are now creating a benchmark an d training data for LLM’s handling legal text\, hoping to spur improvement s.
\n X-TAGS;LANGUAGE=en-US:2023\,Blair-Stanek\,October END:VEVENT BEGIN:VEVENT UID:ai1ec-24005@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nLarge-scale generative models such as GPT and DALL-E have revolutionized natural language processing and computer vision resear ch. These models not only generate high fidelity text or image outputs\, b ut also demonstrate impressive domain and task generalization capabilities . In contrast\, audio generative models are relatively primitive in scale and generalization.\nIn this talk\, I will start with a brief introduction on conventional neural speech generative models and discuss why they are unfit for scaling to Internet-scale data. Next\, by reviewing the latest l arge-scale generative models for text and image\, I will outline a few lin es of promising approaches to build scalable speech models. Last\, I will present Voicebox\, our latest work to advance this area. Voicebox is the m ost versatile generative model for speech. It is trained with a simple tas k — text conditioned speech infilling — on over 50K hours of multilingual speech with a powerful flow-matching objective. Through in-context learnin g\, Voicebox can perform monolingual/cross-lingual zero-shot TTS\, holisti c style conversion\, transient noise removal\, content editing\, and diver se sample generation. Moreover\, Voicebox achieves state-of-the-art perfor mance and excellent run-time efficiency.\nBiography\nWei-Ning Hsu is a res earch scientist at Meta Foundational AI Research (FAIR) and currently the lead of the audio generation team. His research focuses on self-supervised learning and generative models for speech and audio. His pioneering work includes HuBERT\, AV-HuBERT\, TextlessNLP\, data2vec\, wav2vec-U\, textles s speech translation\, and Voicebox. \nPrior to joining Meta\, Wei-Ning wo rked at MERL and Google Brain as a research intern. He received his Ph.D. and S.M. degrees in Electrical Engineering and Computer Science from Massa chusetts Institute of Technology in 2020 and 2018\, under the supervision of Dr. James Glass. He received his B.S. degree in Electrical Engineering from National Taiwan University in 2014\, under the supervision of Prof. L in-shan Lee and Prof. Hsuan-Tien Lin. DTSTART;TZID=America/New_York:20231009T120000 DTEND;TZID=America/New_York:20231009T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Wei-Ning Hsu (Meta Foundational AI Research) “Large Scale Universal Speech Generative Models” URL:https://www.clsp.jhu.edu/events/wei-ning-hsu-meta-foundational-ai-resea rch-large-scale-universal-speech-generative-models/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nLarge-scale generative models such as GPT and DALL-E have revolutionized natural langu age processing and computer vision research. These models not only generat e high fidelity text or image outputs\, but also demonstrate impressive do main and task generalization capabilities. In contrast\, audio generative models are relatively primitive in scale and generalization.
\nIn this talk\, I will st art with a brief introduction on conventional neural speech generative mod els and discuss why they are unfit for scaling to Internet-scale data. Nex t\, by reviewing the latest large-scale generative models for text and ima ge\, I will outline a few lines of promising approaches to build scalable speech models. Last\, I will present Voicebox\, our latest work to advance this area. Voicebox is the most versatile generative model for speech. It is trained with a simple task — text conditioned speech infilling — on ov er 50K hours of multilingual speech with a powerful flow-matching objectiv e. Through in-context learning\, Voicebox can perform monolingual/cross-li ngual zero-shot TTS\, holistic style conversion\, transient noise removal\ , content editing\, and diverse sample generation. Moreover\, Voicebox ach ieves state-of-the-art performance and excellent run-time efficiency.
\nBiography
\nWei-Ning Hsu is a research scientist at Meta Founda tional AI Research (FAIR) and currently the lead of the audio generation t eam. His research focuses on self-supervised learning and generative model s for speech and audio. His pioneering work includes HuBERT\, AV-HuBERT\, TextlessNLP\, data2vec\, wav2vec-U\, textless speech translation\, and Voi cebox.
\nPri or to joining Meta\, Wei-Ning worked at MERL and Google Brain as a researc h intern. He received his Ph.D. and S.M. degrees in Electrical Engineering and Computer Science from Massachusetts Institute of Technology in 2020 a nd 2018\, under the supervision of Dr. James Glass. He received his B.S. d egree in Electrical Engineering from National Taiwan University in 2014\, under the supervision of Prof. Lin-shan Lee and Prof. Hsuan-Tien Lin.
\n X-TAGS;LANGUAGE=en-US:2023\,Hsu\,October END:VEVENT BEGIN:VEVENT UID:ai1ec-23902@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nPretrained language models (LMs) encode implicit repr esentations of knowledge in their parameters. Despite this observation\, o ur best methods for interpreting these representations yield few actionabl e insights on how to manipulate this parameter space for downstream benefi t. In this talk\, I will present work on methods that simulate machine rea soning by localizing and modifying parametric knowledge representations. F irst\, I will present a method for discovering knowledge-critical subnetwo rks within pretrained language models\, and show that these sparse computa tional subgraphs are responsible for the model’s ability to encode specifi c pieces of knowledge. Then\, I will present a new reasoning algorithm\, R ECKONING\, a bi-level optimisation procedure that dynamically encodes and reasons over new knowledge at test-time using the model’s existing learned knowledge representations as a scratchpad. Finally\, I will discuss next steps and challenges in using internal model mechanisms for reasoning.\n\n Bio\n\nAntoine Bosselut is an assistant professor in the School of Compute r and Communication Sciences at the École Polytechnique Fédéral de Lausann e (EPFL). He was a postdoctoral scholar at Stanford University and a Young Investigator at the Allen Institute for AI (AI2). He completed his PhD at the University of Washington and was a student researcher at Microsoft Re search. His research interests are in building systems that mix knowledge and language representations to solve problems in NLP\, specializing in co mmonsense representation and reasoning. DTSTART;TZID=America/New_York:20231013T120000 DTEND;TZID=America/New_York:20231013T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Antoine Bosselut (EPFL) “From Mechanistic Interpretability to Mecha nistic Reasoning” URL:https://www.clsp.jhu.edu/events/antoine-bosselut-epfl/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nAbstr act
\nRecent advances in speech technology make heavy use o f pre-trained models that learn from large quantities of raw (untranscribe d) speech\, using “self-supervised” (ie unsupervised) learning. These mode ls learn to transform the acoustic input into a different representational format that makes supervised learning (for tasks such as transcription or even translation) much easier. However\, *what* and *how* speech-relevant information is encoded in these representations is not well understood. I will talk about some work at various stages of completion in which my gro up is analyzing the structure of these representations\, to gain a more sy stematic understanding of how word-level\, phonetic\, and speaker informat ion is encoded.
\nBiography
\nSharon Goldwate
r is a Professor in the Institute for Language\, Cognition and Computation
at the University of Edinburgh’s School of Informatics. She received her
PhD in 2007 from Brown University and spent two years as a postdoctoral re
searcher at Stanford University before moving to Edinburgh. Her research i
nterests include unsupervised and minimally-supervised learning for speech
and language processing\, computer modelling of language acquisition in c
hildren\, and computational studies of language use. Her main focus withi
n linguistics has been on the lower levels of structure including phonetic
s\, phonology\, and morphology.
Prof. Goldwater has received awards including the 2016 Roger Needha
m Award from the British Computer Society for “distinguished research cont
ribution in computer science by a UK-based researcher who has completed up
to 10 years of post-doctoral research.” She has served on the editorial b
oards of several journals\, including Computational Linguistics\, Transact
ions of the Association for Computational Linguistics\, and the inaugural
board of OPEN MIND: Advances in Cognitive Science. She was a program chair
for the EACL 2014 Conference and chaired the EACL governing board from 20
19-2020.
Abstr act
\nAbstr act
\nMultilingual machine translation has proven immensely useful for both parameter efficiency and overall perf ormance for many language pairs via complete parameter sharing. However\, some language pairs in multilingual models can see worse performance than in bilingual models\, especially in the one-to-many translation setting. M otivated by their empirical differences\, we examine the geometric differe nces in representations from bilingual models versus those from one-to-man y multilingual models. Specifically\, we measure the isotropy of these rep resentations using intrinsic dimensionality and IsoScore\, in order to mea sure how these representations utilize the dimensions in their underlying vector space. We find that for a given language pair\, its multilingual mo del decoder representations are consistently less isotropic than comparabl e bilingual model decoder representations. Additionally\, we show that muc h of this anisotropy in multilingual decoder representations can be attrib uted to modeling language-specific information\, therefore limiting remain ing representational capacity.
\n X-TAGS;LANGUAGE=en-US:2023\,November\,Verma END:VEVENT BEGIN:VEVENT UID:ai1ec-24157@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nIn this talk\, I will present a simple extension of i mage-based Masked Autoencoders (MAE) to self-supervised representation lea rning from audio spectrograms. Following the Transformer encoder-decoder d esign in MAE\, our Audio-MAE first encodes audio spectrogram patches with a high masking ratio\, feeding only the non-masked tokens through encoder layers. The decoder then re-orders and decodes the encoded context padded with mask tokens\, in order to reconstruct the input spectrogram. We find it beneficial to incorporate local window attention in the decoder\, as au dio spectrograms are highly correlated in local time and frequency bands. We then fine-tune the encoder with a lower masking ratio on target dataset s. Empirically\, Audio-MAE sets new state-of-the-art performance on six au dio and speech classification tasks\, outperforming other recent models th at use external supervised pre-training.\nBio\nFlorian Metze is a Research Scientist Manager at Meta AI in New York\, supporting a team of researche rs and engineers working on multi-modal (image\, video\, audio\, text) con tent understanding for Meta’s Family of Apps (Instagram\, Threads\, Facebo ok\, WhatsApp). He used to be an Associate Research Professor at Carnegie Mellon University\, in the School of Computer Science’s Language Technolog ies Institute\, where he still is an Adjunct Professor. He is also a co-fo under of Abridge\, a company working on extracting information from doctor patient conversations. His work covers many areas of speech recognition a nd multi-media analysis with a focus on end-to-end deep learning. Currentl y\, he focuses on multi-modal processing of videos\, and using that inform ation to recommend unconnected content. In the past\, he has worked on low resource and multi-lingual speech processing\, speech recognition with ar ticulatory features\, large-scale multi-media retrieval and summarization\ , information extraction from medical interviews\, and recognition of pers onality or similar meta-data from speech.\nFor more information\, please s ee http://www.cs.cmu.edu/directory/fmetze\n DTSTART;TZID=America/New_York:20231110T120000 DTEND;TZID=America/New_York:20231110T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Florian Metze (CMU) “Masked Autoencoders that Listen” URL:https://www.clsp.jhu.edu/events/florian-metze-cmu/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nIn this talk\, I will present a simple extension of i mage-based Masked Autoencoders (MAE) to self-supervised representation lea rning from audio spectrograms. Following the Transformer encoder-decoder d esign in MAE\, our Audio-MAE first encodes audio spectrogram patches with a high masking ratio\, feeding only the non-masked tokens through encoder layers. The decoder then re-orders and decodes the encoded context padded with mask tokens\, in order to reconstruct the input spectrogram. We find it beneficial to incorporate local window attention in the decoder\, as au dio spectrograms are highly correlated in local time and frequency bands. We then fine-tune the encoder with a lower masking ratio on target dataset s. Empirically\, Audio-MAE sets new state-of-the-art performance on six au dio and speech classification tasks\, outperforming other recent models th at use external supervised pre-training.
\nBio
\nFlorian Metze is a Research Scientist Manager at Meta AI in New York\ , supporting a team of researchers and engineers working on multi-modal (i mage\, video\, audio\, text) content understanding for Meta’s Family of Ap ps (Instagram\, Threads\, Facebook\, WhatsApp). He used to be an Associate Research Professor at Carnegie Mellon University\, in the School of Compu ter Science’s Language Technologies Institute\, where he still is an Adjun ct Professor. He is also a co-founder of Abridge\, a company working on ex tracting information from doctor patient conversations. His work covers ma ny areas of speech recognition and multi-media analysis with a focus on en d-to-end deep learning. Currently\, he focuses on multi-modal processing o f videos\, and using that information to recommend unconnected content. In the past\, he has worked on low resource and multi-lingual speech process ing\, speech recognition with articulatory features\, large-scale multi-me dia retrieval and summarization\, information extraction from medical inte rviews\, and recognition of personality or similar meta-data from speech.< /p>\n
For more information\, please see http://www.cs.cmu.edu/directory/fmetze
\n\n X-TAGS;LANGUAGE=en-US:2023\,Metze\,November END:VEVENT BEGIN:VEVENT UID:ai1ec-24159@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION: DTSTART;TZID=America/New_York:20231113T120000 DTEND;TZID=America/New_York:20231113T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Student Seminar – Kate Sanders URL:https://www.clsp.jhu.edu/events/student-seminar-kate-sanders/ X-COST-TYPE:free X-TAGS;LANGUAGE=en-US:2023\,November\,Sanders END:VEVENT BEGIN:VEVENT UID:ai1ec-24163@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nThe almost unlimited multimedia content available on video-sharing websites has opened new challenges and opportunities for bui lding robust multimodal solutions. This seminar will describe our novel mu ltimodal architectures that (1) are robust to missing modalities\, (2) can identify noisy or less discriminative features\, and (3) can leverage unl abeled data. First\, we present a strategy that effectively combines auxil iary networks\, a transformer architecture\, and an optimized training mec hanism for handling missing features. This problem is relevant since it is expected that during inference the multimodal system will face cases with missing features due to noise or occlusion. We implement this approach fo r audiovisual emotion recognition achieving state-of-the-art performance. Second\, we present a multimodal framework for dealing with scenarios char acterized by noisy or less discriminative features. This situation is comm only observed in audiovisual automatic speech recognition (AV-ASR) with cl ean speech\, where the performance often drops compared to a speech-only s olution due to the variability of visual features. The proposed approach i s a deep learning solution with a gating layer that diminishes the effect of noisy or uninformative visual features\, keeping only useful informatio n. The approach improves\, or at least\, maintains performance when visual features are used. Third\, we discuss alternative strategies to leverage unlabeled multimodal data. A promising approach is to use multimodal prete xt tasks that are carefully designed to learn better representations for p redicting a given task\, leveraging the relationship between acoustic and facial features. Another approach is using multimodal ladder networks wher e intermediate representations are predicted across modalities using later al connections. These models offer principled solutions to increase the ge neralization and robustness of common speech-processing tasks when using m ultimodal architectures. \nBio\nCarlos Busso is a Professor at the Univers ity of Texas at Dallas’s Electrical and Computer Engineering Department\, where he is also the director of the Multimodal Signal Processing (MSP) La boratory. His research interest is in human-centered multimodal machine in telligence and application\, with a focus on the broad areas of affective computing\, multimodal human-machine interfaces\, in-vehicle active safety systems\, and machine learning methods for multimodal processing. He has worked on audio-visual emotion recognition\, analysis of emotional modulat ion in gestures and speech\, designing realistic human-like virtual charac ters\, and detection of driver distractions. He is a recipient of an NSF C AREER Award. In 2014\, he received the ICMI Ten-Year Technical Impact Awar d. In 2015\, his student received the third prize IEEE ITSS Best Dissertat ion Award (N. Li). He also received the Hewlett Packard Best Paper Award a t the IEEE ICME 2011 (with J. Jain)\, and the Best Paper Award at the AAAC ACII 2017 (with Yannakakis and Cowie). He received the Best of IEEE Trans actions on Affective Computing Paper Collection in 2021 (with R. Lotfian) and the Best Paper Award from IEEE Transactions on Affective Computing in 2022 (with Yannakakis and Cowie). He received the ACM ICMI Community Servi ce Award in 2023. In 2023\, he received the Distinguished Alumni Award in the Mid-Career/Academia category by the Signal and Image Processing Instit ute (SIPI) at the University of Southern California. He is currently servi ng as an associate editor of the IEEE Transactions on Affective Computing. He is an IEEE Fellow. He is a member of the ISCA\, and AAAC and a senior member of ACM. DTSTART;TZID=America/New_York:20231117T120000 DTEND;TZID=America/New_York:20231117T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Carlos Busso (University of Texas at Dallas) “Multimodal Machine Le arning for Human-Centric Tasks” URL:https://www.clsp.jhu.edu/events/carl-busso-university-of-texas-at-dalla s-multimodal-machine-learning-for-human-centric-tasks/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\n
Abstr act
\nThe almost unlimited multimedia content available on video-sharing websites has opened new challenges and opportun ities for building robust multimodal solutions. This seminar will describe our novel multimodal architectures that (1) are robust to missing modalit ies\, (2) can identify noisy or less discriminative features\, and (3) can leverage unlabeled data. First\, we present a strategy that effectively c ombines auxiliary networks\, a transformer architecture\, and an optimized training mechanism for handling missing features. This problem is relevan t since it is expected that during inference the multimodal system will fa ce cases with missing features due to noise or occlusion. We implement thi s approach for audiovisual emotion recognition achieving state-of-the-art performance. Second\, we present a multimodal framework for dealing with s cenarios characterized by noisy or less discriminative features. This situ ation is commonly observed in audiovisual automatic speech recognition (AV -ASR) with clean speech\, where the performance often drops compared to a speech-only solution due to the variability of visual features. The propos ed approach is a deep learning solution with a gating layer that diminishe s the effect of noisy or uninformative visual features\, keeping only usef ul information. The approach improves\, or at least\, maintains performanc e when visual features are used. Third\, we discuss alternative strategies to leverage unlabeled multimodal data. A promising approach is to use mul timodal pretext tasks that are carefully designed to learn better represen tations for predicting a given task\, leveraging the relationship between acoustic and facial features. Another approach is using multimodal ladder networks where intermediate representations are predicted across modalitie s using lateral connections. These models offer principled solutions to in crease the generalization and robustness of common speech-processing tasks when using multimodal architectures.
\nBio
\nCarlos Busso is a Professor at the University of Tex as at Dallas’s Electrical and Computer Engineering Department\, where he i s also the director of the Multimodal Signal Processing (MSP) Laboratory. His research interest is in human-centered multimodal machine intelligence and application\, with a focus on the broad areas of affective computing\ , multimodal human-machine interfaces\, in-vehicle active safety systems\, and machine learning methods for multimodal processing. He has worked on audio-visual emotion recognition\, analysis of emotional modulation in ges tures and speech\, designing realistic human-like virtual characters\, and detection of driver distractions. He is a recipient of an NSF CAREER Awar d. In 2014\, he received the ICMI Ten-Year Technical Impact Award. In 2015 \, his student received the third prize IEEE ITSS Best Dissertation Award (N. Li). He also received the Hewlett Packard Best Paper Award at the IEEE ICME 2011 (with J. Jain)\, and the Best Paper Award at the AAAC ACII 2017 (with Yannakakis and Cowie). He received the Best of IEEE Transactions on Affective Computing Paper Collection in 2021 (with R. Lotfian) and the Be st Paper Award from IEEE Transactions on Affective Computing in 2022 (with Yannakakis and Cowie). He received the ACM ICMI Community Service Award i n 2023. In 2023\, he received the Distinguished Alumni Award in the Mid-Ca reer/Academia category by the Signal and Image Processing Institute (SIPI) at the University of Southern California. He is currently serving as an a ssociate editor of the IEEE Transactions on Affective Computing. He is an IEEE Fellow. He is a member of the ISCA\, and AAAC and a senior member of ACM.
\n X-TAGS;LANGUAGE=en-US:2023\,Busso\,November END:VEVENT BEGIN:VEVENT UID:ai1ec-24165@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION: DTSTART;TZID=America/New_York:20231127T120000 DTEND;TZID=America/New_York:20231127T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Student Seminar – Aleem Khan URL:https://www.clsp.jhu.edu/events/student-seminar-aleem-khan/ X-COST-TYPE:free X-TAGS;LANGUAGE=en-US:2023\,Khan\,November END:VEVENT BEGIN:VEVENT UID:ai1ec-24167@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nPre-trained speech representation models have become ubiquitous in speech processing over the past few years. They have both i mproved the state of the art and made it feasible to learn task-specific m odels with very little labeled data. However\, it is not well understood what linguistic information is encoded in pre-trained models and how best to apply them to downstream tasks. In this talk I will describe recent wor k that begins to build an understanding of the layer-wise information lear ned by pre-trained speech models. We consider a number of popular pre-tra ined models and investigate the extent to which their layers encode spectr al\, phonetic\, and word-level information. The results of these analyses also suggest some ways to improve or simplify the application of pre-trai ned models for downstream tasks. Finally\, I will describe our efforts to benchmark model performance on a variety of spoken language understanding tasks\, in order to broaden our understanding of the capabilities of stat e-of-the-art models.\nThis talk is based in part on work presented in\nA. Pasad et al.\, “Comparative layer-wise analysis of self-supervised speech models\,”ICASSP 2023.\nA. Pasad et al.\, “What do self-supervised speech m odels know about words?\,” arXiv:2307.00162\, 2023.\nS. Shon et al.\, “SLU E Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Task s\,” ACL 2023.\nBio\nKaren Livescu is a Professor at TTI-Chicago. She comp leted her PhD at MIT in 2005. She is an ISCA Fellow and a recent IEEE Dist inguished Lecturer. She has served as a program chair/co-chair for ICLR\, Interspeech\, and ASRU\, and is an Associate Editor for TACL and IEEE T-P AMI. Her group’s work spans a variety of topics in spoken\, written\, and signed language processing. DTSTART;TZID=America/New_York:20231201T120000 DTEND;TZID=America/New_York:20231201T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Karen Livescu (Toyota Technological Institute at Chicago) “What Do Pre-Trained Speech Representation Models Know? Layer-Wise Analysis and Ben chmarking” URL:https://www.clsp.jhu.edu/events/karen-livescu-toyota-technological-inst itute-at-chicago/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nPre-trained speech representation models have become ubiquitous in speech processing over the past few years. They have both i mproved the state of the art and made it feasible to learn task-specific m odels with very little labeled data. However\, it is not well understood what linguistic information is encoded in pre-trained models and how best to apply them to downstream tasks. In this talk I will describe recent wor k that begins to build an understanding of the layer-wise information lear ned by pre-trained speech models. We consider a number of popular pre-tra ined models and investigate the extent to which their layers encode spectr al\, phonetic\, and word-level information. The results of these analyses also suggest some ways to improve or simplify the application of pre-trai ned models for downstream tasks. Finally\, I will describe our efforts to benchmark model performance on a variety of spoken language understanding tasks\, in order to broaden our understanding of the capabilities of stat e-of-the-art models.
\nThis talk is based in part on work presented in
\nA. Pasad et al.\, “C omparative layer-wise analysis of self-supervised speech models\,”ICAS SP 2023.
\nA. Pasad et al.\, “What do self-supervised speech models know about words?\,” ar Xiv:2307.00162\, 2023.
\nS. Shon et al.\, “SLUE Phase-2: A Benchmark Suite of Diverse Spo ken Language Understanding Tasks\,” ACL 2023.
\nBio
\nKaren Livescu is a Professor at TTI-Chicago. She completed he r PhD at MIT in 2005. She is an ISCA Fellow and a recent IEEE Distinguishe d Lecturer. She has served as a program chair/co-chair for ICLR\, Intersp eech\, and ASRU\, and is an Associate Editor for TACL and IEEE T-PAMI. He r group’s work spans a variety of topics in spoken\, written\, and signed language processing.
\n X-TAGS;LANGUAGE=en-US:2023\,December\,Livescu END:VEVENT BEGIN:VEVENT UID:ai1ec-24169@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nFoundation models\, including Chat-GPT and its many v ariants\, have come into prominence in the natural language processing (NL P) community thanks the ubiquity of text data readily available on the int ernet and the design of modern transformer architectures that can effectiv ely learn from such data. However\, the development of a foundation model for sequential decision-making (e.g.\, reinforcement learning\, planning) is faced with additional challenges not present in NLP. In this talk\, we discuss some of these challenges with the hope of informing future investm ents that funding agencies and the academic community should engage in. Th e problem of transfer learning in the context of sequential decision-makin g is also discussed and constitutes one of the challenges that foundation models must address.\nBio\nAlvaro Velasquez a program manager at the Defen se Advanced Research Projects Agency (DARPA)\, where he currently leads pr ograms on neuro-symbolic AI. Before that\, Alvaro oversaw the machine inte lligence portfolio for the Information Directorate of the Air Force Resear ch Laboratory (AFRL). Alvaro is a recipient of the distinguished paper awa rd from AAAI and best paper and patent awards from AFRL\, the National Sci ence Foundation Graduate Research Fellowship. He has authored over 70 pape rs and two patents and serves as Associate Editor of the IEEE Transactions on Artificial Intelligence. DTSTART;TZID=America/New_York:20231204T120000 DTEND;TZID=America/New_York:20231204T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Alvaro Velasquez (DARPA) “Foundation Models and the Transfer of Emb odied Autonomy” URL:https://www.clsp.jhu.edu/events/alvaro-velasquez/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nFoundation models\, including Chat-GPT and its many v ariants\, have come into prominence in the natural language processing (NL P) community thanks the ubiquity of text data readily available on the int ernet and the design of modern transformer architectures that can effectiv ely learn from such data. However\, the development of a foundation model for sequential decision-making (e.g.\, reinforcement learning\, planning) is faced with additional challenges not present in NLP. In this talk\, we discuss some of these challenges with the hope of informing future investm ents that funding agencies and the academic community should engage in. Th e problem of transfer learning in the context of sequential decision-makin g is also discussed and constitutes one of the challenges that foundation models must address.
\nBio
\nAlvaro Velasquez a program manager at the Defense Advanced Research Pr ojects Agency (DARPA)\, where he currently leads programs on neuro-symboli c AI. Before that\, Alvaro oversaw the machine intelligence portfolio for the Information Directorate of the Air Force Research Laboratory (AFRL). A lvaro is a recipient of the distinguished paper award from AAAI and best p aper and patent awards from AFRL\, the National Science Foundation Graduat e Research Fellowship. He has authored over 70 papers and two patents and serves as Associate Editor of the IEEE Transactions on Artificial Intellig ence.
\n X-TAGS;LANGUAGE=en-US:2023\,December\,Velasquez END:VEVENT BEGIN:VEVENT UID:ai1ec-24511@www.clsp.jhu.edu DTSTAMP:20240329T113627Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION: DTSTART;TZID=America/New_York:20240412T120000 DTEND;TZID=America/New_York:20240412T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Sonal Joshi (JHU) URL:https://www.clsp.jhu.edu/events/sonal-joshi-jhu/ X-COST-TYPE:free X-TAGS;LANGUAGE=en-US:2024\,April\,Joshi END:VEVENT END:VCALENDAR