BEGIN:VCALENDAR VERSION:2.0 PRODID:-//128.220.36.25//NONSGML kigkonsult.se iCalcreator 2.26.9// CALSCALE:GREGORIAN METHOD:PUBLISH X-FROM-URL:https://www.clsp.jhu.edu X-WR-TIMEZONE:America/New_York BEGIN:VTIMEZONE TZID:America/New_York X-LIC-LOCATION:America/New_York BEGIN:STANDARD DTSTART:20231105T020000 TZOFFSETFROM:-0400 TZOFFSETTO:-0500 RDATE:20241103T020000 TZNAME:EST END:STANDARD BEGIN:DAYLIGHT DTSTART:20240310T020000 TZOFFSETFROM:-0500 TZOFFSETTO:-0400 RDATE:20250309T020000 TZNAME:EDT END:DAYLIGHT END:VTIMEZONE BEGIN:VEVENT UID:ai1ec-20117@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nNeural sequence generation systems oftentimes generat e sequences by searching for the most likely sequence under the learnt pro bability distribution. This assumes that the most likely sequence\, i.e. t he mode\, under such a model must also be the best sequence it has to offe r (often in a given context\, e.g. conditioned on a source sentence in tra nslation). Recent findings in neural machine translation (NMT) show that t he true most likely sequence oftentimes is empty under many state-of-the-a rt NMT models. This follows a large list of other pathologies and biases o bserved in NMT and other sequence generation models: a length bias\, large r beams degrading performance\, exposure bias\, and many more. Many of the se works blame the probabilistic formulation of NMT or maximum likelihood estimation. We provide a different view on this: it is mode-seeking search \, e.g. beam search\, that introduces many of these pathologies and biases \, and such a decision rule is not suitable for the type of distributions learnt by NMT systems. We show that NMT models spread probability mass ove r many translations\, and that the most likely translation oftentimes is a rare event. We further show that translation distributions do capture imp ortant aspects of translation well in expectation. Therefore\, we advocate for decision rules that take into account the entire probability distribu tion and not just its mode. We provide one example of such a decision rule \, and show that this is a fruitful research direction.\nBiography\nI am a n assistant professor (UD) in natural language processing at the Institute for Logic\, Language and Computation where I lead the Probabilistic Langu age Learning group.\nMy work concerns the design of models and algorithms that learn to represent\, understand\, and generate language data. Example s of specific problems I am interested in include language modelling\, mac hine translation\, syntactic parsing\, textual entailment\, text classific ation\, and question answering.\nI also develop techniques to approach gen eral machine learning problems such as probabilistic inference\, gradient and density estimation.\nMy interests sit at the intersection of disciplin es such as statistics\, machine learning\, approximate inference\, global optimization\, formal languages\, and computational linguistics.\n \n DTSTART;TZID=America/New_York:20210419T120000 DTEND;TZID=America/New_York:20210419T131500 LOCATION:via Zoom SEQUENCE:0 SUMMARY:Wilker Aziz (University of Amsterdam) “The Inadequacy of the Mode in Neural Machine Translation” URL:https://www.clsp.jhu.edu/events/wilker-aziz-university-of-amsterdam/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n
\\nAbstr act
\nNeural sequence generation systems oftentimes generat e sequences by searching for the most likely sequence under the learnt pro bability distribution. This assumes that the most likely sequence\, i.e. t he mode\, under such a model must also be the best sequence it has to offe r (often in a given context\, e.g. conditioned on a source sentence in tra nslation). Recent findings in neural machine translation (NMT) show that t he true most likely sequence oftentimes is empty under many state-of-the-a rt NMT models. This follows a large list of other pathologies and biases o bserved in NMT and other sequence generation models: a length bias\, large r beams degrading performance\, exposure bias\, and many more. Many of the se works blame the probabilistic formulation of NMT or maximum likelihood estimation. We provide a different view on this: it is mode-seeking search \, e.g. beam search\, that introduces many of these pathologies and biases \, and such a decision rule is not suitable for the type of distributions learnt by NMT systems. We show that NMT models spread probability mass ove r many translations\, and that the most likely translation oftentimes is a rare event. We further show that translation distributions do capture imp ortant aspects of translation well in expectation. Therefore\, we advocate for decision rules that take into account the entire probability distribu tion and not just its mode. We provide one example of such a decision rule \, and show that this is a fruitful research direction.
\nBi ography
\nI am an assistant professor (UD) in natu ral language processing at the Institute for Logic\, Language and Computation where I lead the Probabilistic Language Learning group.
\nMy work concerns the design of models and algorithms that learn to represe nt\, understand\, and generate language data. Examples of specific problem s I am interested in include language modelling\, machine translation\, sy ntactic parsing\, textual entailment\, text classification\, and question answering.
\nI also develop techniques to approach general machine l earning problems such as probabilistic inference\, gradient and density es timation.
\nMy interests sit at the intersection of disciplines such as statistics\, machine learning\, approximate inference\, global optimiz ation\, formal languages\, and computational linguistics.
\n\n< p> \n X-TAGS;LANGUAGE=en-US:2021\,April\,Aziz END:VEVENT BEGIN:VEVENT UID:ai1ec-20120@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nRobotics@Google’s mission is to make robots useful in the real world through machine learning. We are excited about a new model for robotics\, designed for generalization across diverse environments an d instructions. This model is focused on scalable data-driven learning\, w hich is task-agnostic\, leverages simulation\, learns from past experience \, and can be quickly adapted to work in the real-world through limited in teractions. In this talk\, we’ll share some of our recent work in this dir ection in both manipulation and locomotion applications.\nBiography\nCarol ina Parada is a Senior Engineering Manager at Google Robotics. She leads t he robot-mobility group\, which focuses on improving robot motion planning \, navigation\, and locomotion\, using reinforcement learning. Prior to th at\, she led the camera perception team for self-driving cars at Nvidia fo r 2 years. She was also a lead with Speech @ Google for 7 years\, where sh e drove multiple research and engineering efforts that enabled Ok Google\, the Google Assistant\, and Voice-Search. Carolina grew up in Venezuela an d moved to the US to pursue a B.S. and M.S. degree in Electrical Engineeri ng at University of Washington and her Phd at Johns Hopkins University at the Center for Language and Speech Processing (CLSP). DTSTART;TZID=America/New_York:20210423T120000 DTEND;TZID=America/New_York:20210423T131500 LOCATION:via Zoom SEQUENCE:0 SUMMARY:Carolina Parada (Google AI) “State of Robotics @ Google” URL:https://www.clsp.jhu.edu/events/carolina-parada-google-ai/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\n
Abstr act
\nRobotics@Google’s mission is to make robots useful i n the real world through machine learning. We are excited about a new mode l for robotics\, designed for generalization across diverse environments a nd instructions. This model is focused on scalable data-driven learning\, which is task-agnostic\, leverages simulation\, learns from past experienc e\, and can be quickly adapted to work in the real-world through limited i nteractions. In this talk\, we’ll share some of our recent work in this di rection in both manipulation and locomotion applications.
\n< strong>Biography
\nCarolina Parad a is a Senior Engineering Manager at Google Robotics. She leads the robot-mobility group\, which focuses on improving robot motion planning\, navigation\, and locomotion\, using reinforcement learning. Prior to that \, she led the camera perception team for self-driving cars at Nvidia for 2 years. She was also a lead with Speech @ Google for 7 years\, where she drove multiple research and engineering efforts that enabled Ok Google\, t he Google Assistant\, and Voice-Search. Carolina< /span> grew up in Venezuela and moved to the US to pursue a B.S. and M.S. degree in Electrical Engineering at University of Washington and her Phd a t Johns Hopkins University at the Center for Language and Speech Processin g (CLSP).
\n X-TAGS;LANGUAGE=en-US:2021\,April\,Parada END:VEVENT BEGIN:VEVENT UID:ai1ec-21267@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nIn this talk\, I present a multipronged strategy for zero-shot cross-lingual Information Extraction\, that is the construction of an IE model for some target language\, given existing annotations exclu sively in some other language. This work is part of the JHU team’s effort under the IARPA BETTER program. I explore data augmentation techniques inc luding data projection and self-training\, and how different pretrained en coders impact them. We find through extensive experiments and extension of techniques that a combination of approaches\, both new and old\, leads to better performance than any one cross-lingual strategy in particular.\nBi ography\nMahsa Yarmohammadi is an assistant research scientist in CLSP\, J HU\, who leads state-of-the-art research in cross-lingual language and spe ech applications and algorithms. A primary focus of Yarmohammadi’s researc h is using deep learning techniques to transfer existing resources into ot her languages and to learn representations of language from multilingual d ata. She also works in automatic speech recognition and speech translation . Yarmohammadi received her PhD in computer science and engineering from O regon Health & Science University (2016). She joined CLSP as a post-doctor al fellow in 2017. DTSTART;TZID=America/New_York:20220204T120000 DTEND;TZID=America/New_York:20220204T131500 LOCATION:Ames 234 Presented Virtually via Zoom https://wse.zoom.us/j/967351 83473 SEQUENCE:0 SUMMARY:Mahsa Yarmohammadi (Johns Hopkins University) “Data Augmentation fo r Zero-shot Cross-Lingual Information Extraction” URL:https://www.clsp.jhu.edu/events/mahsa-yarmohammadi-johns-hopkins-univer sity-data-augmentation-for-zero-shot-cross-lingual-information-extraction/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nIn this talk\, I present a multipronged strategy for zero-shot cross-lingual Information Extraction\, that is the construction of an IE model for some target language\, given existing annotations exclu sively in some other language. This work is part of the JHU team’s effort under the IARPA BETTER program. I explore data augmentation techniques inc luding data projection and self-training\, and how different pretrained en coders impact them. We find through extensive experiments and extension of techniques that a combination of approaches\, both new and old\, leads to better performance than any one cross-lingual strategy in particular.
\nBiography
\nAbstr act
\nSocial media allows researchers to track societal and cultural changes over time based on language analysis tools. Many of thes e tools rely on statistical algorithms which need to be tuned to specific types of language. Recent studies have questioned the robustness of longit udinal analyses based on statistical methods due to issues of temporal bia s and semantic shift. To what extent are changes in semantics over time af fecting the reliability of longitudinal analyses? We examine this question through a case study: understanding shifts in mental health during the co urse of the COVID-19 pandemic. We demonstrate that a recently-introduced m ethod for measuring semantic shift may be used to proactively identify fai lure points of language-based models and improve predictive generalization over time. Ultimately\, we find that these analyses are critical to produ cing accurate longitudinal studies of social media.
\n X-TAGS;LANGUAGE=en-US:2022\,February\,Harrigian END:VEVENT BEGIN:VEVENT UID:ai1ec-21277@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nAs humans\, our understanding of language is grounded in a rich mental model about “how the world works” – that we learn throug h perception and interaction. We use this understanding to reason beyond w hat we literally observe or read\, imagining how situations might unfold i n the world. Machines today struggle at this kind of reasoning\, which lim its how they can communicate with humans.In my talk\, I will discuss three lines of work to bridge this gap between machines and humans. I will firs t discuss how we might measure grounded understanding. I will introduce a suite of approaches for constructing benchmarks\, using machines in the lo op to filter out spurious biases. Next\, I will introduce PIGLeT: a model that learns physical commonsense understanding by interacting with the wor ld through simulation\, using this knowledge to ground language. From an E nglish-language description of an event\, PIGLeT can anticipate how the wo rld state might change – outperforming text-only models that are orders of magnitude larger. Finally\, I will introduce MERLOT\, which learns about situations in the world by watching millions of YouTube videos with transc ribed speech. Through training objectives inspired by the developmental ps ychology idea of multimodal reentry\, MERLOT learns to fuse language\, vis ion\, and sound together into powerful representations.Together\, these di rections suggest a path forward for building machines that learn language rooted in the world.\nBiography\nRowan Zellers is a final year PhD candida te at the University of Washington in Computer Science & Engineering\, adv ised by Yejin Choi and Ali Farhadi. His research focuses on enabling machi nes to understand language\, vision\, sound\, and the world beyond these m odalities. He has been recognized through an NSF Graduate Fellowship and a NeurIPS 2021 outstanding paper award. His work has appeared in several me dia outlets\, including Wired\, the Washington Post\, and the New York Tim es. In the past\, he graduated from Harvey Mudd College with a B.S. in Com puter Science & Mathematics\, and has interned at the Allen Institute for AI. DTSTART;TZID=America/New_York:20220214T120000 DTEND;TZID=America/New_York:20220214T131500 LOCATION:Ames Hall 234 - Presented Virtually Via Zoom https://wse.zoom.us/j /96735183473 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Rowan Zellers (University of Washington) ” Grounding Language by Se eing\, Hearing\, and Interacting” URL:https://www.clsp.jhu.edu/events/rowan-zellers-university-of-washington- grounding-language-by-seeing-hearing-and-interacting/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nAs humans\, our understanding of language is grounded
in a rich mental model about “how the world works” – that we learn throug
h perception and interaction. We use this understanding to reason beyond w
hat we literally observe or read\, imagining how situations might unfold i
n the world. Machines today struggle at this kind of reasoning\, which lim
its how they can communicate with humans.
In my talk\, I will discuss three lines of work to bridge
this gap between machines and humans. I will first discuss how we might m
easure grounded understanding. I will introduce a suite of approaches for
constructing benchmarks\, using machines in the loop to filter out spuriou
s biases. Next\, I will introduce PIGLeT: a model that learns physical com
monsense understanding by interacting with the world through simulation\,
using this knowledge to ground language. From an English-language descript
ion of an event\, PIGLeT can anticipate how the world state might change –
outperforming text-only models that are orders of magnitude larger. Final
ly\, I will introduce MERLOT\, which learns about situations in the world
by watching millions of YouTube videos with transcribed speech. Through tr
aining objectives inspired by the developmental psychology idea of multimo
dal reentry\, MERLOT learns to fuse language\, vision\, and sound together
into powerful representations.
Together\, these directions suggest a path forward for building mac
hines that learn language rooted in the world.
Biography strong>
\nRowan Zellers is a final year PhD candidate at the Univers ity of Washington in Computer Science & Engineering\, advised by Yejin Cho i and Ali Farhadi. His research focuses on enabling machines to understand language\, vision\, sound\, and the world beyond these modalities. He has been recognized through an NSF Graduate Fellowship and a NeurIPS 2021 out standing paper award. His work has appeared in several media outlets\, inc luding Wired\, the Washington Post\, and the New York Times. In the past\, he graduated from Harvey Mudd College with a B.S. in Computer Science & M athematics\, and has interned at the Allen Institute for AI.
\n< /HTML> X-TAGS;LANGUAGE=en-US:2022\,February\,Zellers END:VEVENT BEGIN:VEVENT UID:ai1ec-21280@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nAs AI-driven language interfaces (such as chat-bots) become more integrated into our lives\, they need to become more versatile and reliable in their communication with human users. How can we make pro gress toward building more “general” models that are capable of understand ing a broader spectrum of language commands\, given practical constraints such as the limited availability of labeled data?\nIn this talk\, I will d escribe my research toward addressing this question along two dimensions o f generality. First I will discuss progress in “breadth” — models that add ress a wider variety of tasks and abilities\, drawing inspiration from exi sting statistical learning techniques such as multi-task learning. In part icular\, I will showcase a system that works well on several QA benchmarks \, resulting in state-of-the-art results on 10 benchmarks. Furthermore\, I will show its extension to tasks beyond QA (such as text generation or cl assification) that can be “defined” via natural language. In the second p art\, I will focus on progress in “depth” — models that can handle complex inputs such as compositional questions. I will introduce Text Modular Net works\, a general framework that casts problem-solving as natural language communication among simpler “modules.” Applying this framework to composi tional questions by leveraging discrete optimization and existing non-comp ositional closed-box QA models results in a model with strong empirical pe rformance on multiple complex QA benchmarks while providing human-readable reasoning.\nI will conclude with future research directions toward broade r NLP systems by addressing the limitations of the presented ideas and oth er missing elements needed to move toward more general-purpose interactive language understanding systems.\nBiography\nDaniel Khashabi is a postdoct oral researcher at the Allen Institute for Artificial Intelligence (AI2)\, Seattle. Previously\, he completed his Ph.D. in Computer and Information Sciences at the University of Pennsylvania in 2019. His interests lie at t he intersection of artificial intelligence and natural language processing \, with a vision toward more general systems through unified algorithms an d theories. DTSTART;TZID=America/New_York:20220218T120000 DTEND;TZID=America/New_York:20220218T131500 LOCATION:Ames Hall 234 - Presented Virtually Via Zoom https://wse.zoom.us/j /96735183473 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Daniel Khashabi (Allen Institute for Artificial Intelligence) “The Quest Toward Generality in Natural Language Understanding” URL:https://www.clsp.jhu.edu/events/daniel-khashabi-allen-institute-for-art ificial-intelligence/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nAs AI-driven language interfaces (such as c hat-bots) become more integrated into our lives\, they need to become more versatile and reliable in their communication with human users. How can w e make progress toward building more “general” models that are capable of understanding a broader spectrum of language commands\, given practical co nstraints such as the limited availability of labeled data?
\nIn this talk\, I will describe my research toward addressing this ques tion along two dimensions of generality. First I will discuss progress in “breadth” — models that address a wider variety of tasks and abilities\, d rawing inspiration from existing statistical learning techniques such as m ulti-task learning. In particular\, I will showcase a system that works we ll on several QA benchmarks\, resulting in state-of-the-art results on 10 benchmarks. Furthermore\, I will show its extension to tasks beyond QA (su ch as text generation or classification) that can be “defined” via natural language. In the second part\, I will focus on progress in “depth” — mod els that can handle complex inputs such as compositional questions. I will introduce Text Modular Networks\, a general framework that casts problem- solving as natural language communication among simpler “modules.” Applyin g this framework to compositional questions by leveraging discrete optimiz ation and existing non-compositional closed-box QA models results in a mod el with strong empirical performance on multiple complex QA benchmarks whi le providing human-readable reasoning.
\nI will conclude w ith future research directions toward broader NLP systems by addressing th e limitations of the presented ideas and other missing elements needed to move toward more general-purpose interactive language understanding system s.
\nBiography
\nDaniel Khashabi is a postdoctoral researcher at the Allen Institute for Artificia l Intelligence (AI2)\, Seattle. Previously\, he completed his Ph.D. in Com puter and Information Sciences at the University of Pennsylvania in 2019. His interests lie at the intersection of artificial intelligence and natur al language processing\, with a vision toward more general systems through unified algorithms and theories.
\n X-TAGS;LANGUAGE=en-US:2022\,February\,Khashabi END:VEVENT BEGIN:VEVENT UID:ai1ec-21487@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nEnormous amounts of ever-changing knowledge are avai lable online in diverse textual styles and diverse formats. Recent advance s in deep learning algorithms and large-scale datasets are spurring progre ss in many Natural Language Processing (NLP) tasks\, including question an swering. Nevertheless\, these models cannot scale up when task-annotated t raining data are scarce. This talk presents my lab’s work toward building general-purpose models in NLP and how to systematically evaluate them. Fir st\, I present a general model for two known tasks of question answering i n English and multiple languages that are robust to small domain shifts. Then\, I show a meta-training approach that can solve a variety of NLP tas ks with only using a few examples and introduce a benchmark to evaluate cr oss-task generalization. Finally\, I discuss neuro-symbolic approaches to address more complex tasks by eliciting knowledge from structured data and language models.\n\nBiography\n\nHanna Hajishirzi is an Assistant Profess or in the Paul G. Allen School of Computer Science & Engineering at the Un iversity of Washington and a Senior Research Manager at the Allen Institut e for AI. Her research spans different areas in NLP and AI\, focusing on d eveloping general-purpose machine learning algorithms that can solve many NLP tasks. Applications for these algorithms include question answering\, representation learning\, green AI\, knowledge extraction\, and conversati onal dialogue. Honors include the NSF CAREER Award\, Sloan Fellowship\, Al len Distinguished Investigator Award\, Intel rising star award\, best pape r and honorable mention awards\, and several industry research faculty awa rds. Hanna received her PhD from University of Illinois and spent a year a s a postdoc at Disney Research and CMU. DTSTART;TZID=America/New_York:20220225T120000 DTEND;TZID=America/New_York:20220225T131500 LOCATION:Ames Hall 234 - Presented Virtually Via Zoom https://wse.zoom.us/j /96735183473 SEQUENCE:0 SUMMARY:Hanna Hajishirzi (University of Washington & Allen Institute for AI ) “Toward Robust\, Knowledge-Rich NLP” URL:https://www.clsp.jhu.edu/events/hanna-hajishirzi-university-of-washingt on-allen-institute-for-ai-toward-robust-knowledge-rich-nlp/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nAbstr act
\nSince it is increasingly harder to opt out from inter acting with AI technology\, people demand that AI is capable of maintainin g contracts such that it supports agency and oversight of people who are r equired to use it or who are affected by it. To help those people create a mental model about how to interact with AI systems\, I extend the underly ing models to self-explain—predict the label/answer and explain this predi ction. In this talk\, I will present how to generate (1) free-text explana tions given in plain English that immediately tell users the gist of the r easoning\, and (2) contrastive explanations that help users understand how they could change the text to get another label.
\nBiograph y
\nAna Marasović is a postdoctoral researcher at the Allen Institute for AI (AI2) and the Paul G. Allen School of Computer Science & Engineering at University of Washington. Her research interests broadly l ie in the fields of natural language processing\, explainable AI\, and vis ion-and-language learning. Her projects are motivated by a unified goal: i mprove interaction and control of the NLP systems to help people make thes e systems do what they want with the confidence that they’re getting exact ly what they need. Prior to joining AI2\, Ana obtained her PhD from Heidel berg University.
\nHow to pronounce my name: the first name i s Ana like in Spanish\, i.e.\, with a long “a” like in “water”\; regarding the last name: “mara” as in actress mara wilson + “so” + “veetch”.
\n< /BODY> X-TAGS;LANGUAGE=en-US:2022\,February\,Marasovic END:VEVENT BEGIN:VEVENT UID:ai1ec-23304@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nTransformers are essential to pretraining. As we appr oach 5 years of BERT\, the connection between attention as architecture an d transfer learning remains key to this central thread in NLP. Other archi tectures such as CNNs and RNNs have been used to replicate pretraining res ults\, but these either fail to reach the same accuracy or require supplem ental attention layers. This work revisits the semanal BERT result and con siders pretraining without attention. We consider replacing self-attention layers with recently developed approach for long-range sequence modeling and transformer architecture variants. Specifically\, inspired by recent p apers like the structured space space sequence model (S4)\, we use simple routing layers based on state-space models (SSM) and a bidirectional model architecture based on multiplicative gating. We discuss the results of th e proposed Bidirectional Gated SSM (BiGS) and present a range of analysis into its properties. Results show that architecture does seem to have a no table impact on downstream performance and a different inductive bias that is worth exploring further.\nBiography\nAlexander “Sasha” Rush is an Asso ciate Professor at Cornell Tech. His work is at the intersection of natura l language processing and generative modeling with applications in text ge neration\, efficient inference\, and controllability. He has written sever al popular open-source software projects supporting NLP research and data science\, and works part-time as a researcher at Hugging Face. He is the s ecretary of ICLR and developed software used to run virtual conferences du ring COVID. His work has received paper and demo awards at major NLP\, vis ualization\, and hardware conferences\, an NSF Career Award\, and a Sloan Fellowship. He tweets and blogs\, mostly about coding and ML\, at @srush_n lp. DTSTART;TZID=America/New_York:20230203T120000 DTEND;TZID=America/New_York:20230203T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Sasha Rush (Cornell University) “Pretraining Without Attention” URL:https://www.clsp.jhu.edu/events/sasha-rush-cornell-university/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nTransformers are essential to pretraining. As we appr oach 5 years of BERT\, the connection between attention as architecture an d transfer learning remains key to this central thread in NLP. Other archi tectures such as CNNs and RNNs have been used to replicate pretraining res ults\, but these either fail to reach the same accuracy or require supplem ental attention layers. This work revisits the semanal BERT result and con siders pretraining without attention. We consider replacing self-attention layers with recently developed approach for long-range sequence modeling and transformer architecture variants. Specifically\, inspired by recent p apers like the structured space space sequence model (S4)\, we use simple routing layers based on state-space models (SSM) and a bidirectional model architecture based on multiplicative gating. We discuss the results of th e proposed Bidirectional Gated SSM (BiGS) and present a range of analysis into its properties. Results show that architecture does seem to have a no table impact on downstream performance and a different inductive bias that is worth exploring further.
\nBiography
\nAbstr act
\nWhile large language models have advanced the state-o f-the-art in natural language processing\, these models are trained on lar ge-scale datasets\, which may include harmful information. Studies have sh own that as a result\, the models exhibit social biases and generate misin formation after training. In this talk\, I will discuss my work on analyzi ng and interpreting the risks of large language models across the areas of fairness\, trustworthiness\, and safety. I will first describe my researc h in the detection of dialect bias between African American English (AAE) vs. Standard American English (SAE). The second part investigates the trus tworthiness of models through the memorization and subsequent generation o f conspiracy theories. I will end my talk with recent work in AI safety re garding text that may lead to physical harm.
\nBiography
\nSharon is a 5th-year Ph.D. candidate at the University of Ca lifornia\, Santa Barbara\, where she is advised by Professor William Wang. Her research interests lie in natural language processing\, with a focus on Responsible AI. Sharon’s research spans the subareas of fairness\, trus tworthiness\, and safety\, with publications in ACL\, EMNLP\, WWW\, and LR EC. She has spent summers interning at AWS\, Meta\, and Pinterest. Sharon is a 2022 EECS Rising Star and a current recipient of the Amazon Alexa AI Fellowship for Responsible AI.
\n X-TAGS;LANGUAGE=en-US:2023\,February\,Levy END:VEVENT BEGIN:VEVENT UID:ai1ec-23308@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nBiases in datasets\, or unintentionally introduced sp urious cues\, are a common source of misspecification in machine learning. Performant models trained on such data can gender stereotype or be brittl e under distribution shift. In this talk\, we present several results in multimodal and question answering applications studying sources of dataset bias\, and several mitigation methods. We propose approaches where known dimensions of dataset bias are explicitly factored out of a model during learning\, without needing to modify data. Finally\, we ask whether datase t biases can be attributable to annotator behavior during annotation. Draw ing inspiration from work in psychology on cognitive biases\, we show cert ain behavioral patterns are highly indicative of the creation of problemat ic (but valid) data instances in question answering. We give evidence that many existing observations around how dataset bias propagates to models c an be attributed to data samples created by annotators we identify.\nBiogr aphy\nMark Yatskar is an Assistant Professor at University of Pennsylvania in the department of Computer and Information Science. He did his PhD at University of Washington co-advised by Luke Zettlemoyer and Ali Farhadi. H e was a Young Investigator at the Allen Institute for Artificial Intellige nce for several years working with their computer vision team\, Prior. His work spans Natural Language Processing\, Computer Vision\, and Fairness i n Machine Learning. He received a Best Paper Award at EMNLP for work on ge nder bias amplification\, and his work has been featured in Wired and the New York Times. DTSTART;TZID=America/New_York:20230210T120000 DTEND;TZID=America/New_York:20230210T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Mark Yatskar (University of Pennsylvania) “Understanding Dataset Bi ases: Behavioral Indicators During Annotation and Contrastive Mitigations” URL:https://www.clsp.jhu.edu/events/mark-yatskar-university-of-pennsylvania / X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nBiases in datasets\, or unintentionally introduced sp urious cues\, are a common source of misspecification in machine learning. Performant models trained on such data can gender stereotype or be brittl e under distribution shift. In this talk\, we present several results in multimodal and question answering applications studying sources of dataset bias\, and several mitigation methods. We propose approaches where known dimensions of dataset bias are explicitly factored out of a model during learning\, without needing to modify data. Finally\, we ask whether datase t biases can be attributable to annotator behavior during annotation. Draw ing inspiration from work in psychology on cognitive biases\, we show cert ain behavioral patterns are highly indicative of the creation of problemat ic (but valid) data instances in question answering. We give evidence that many existing observations around how dataset bias propagates to models c an be attributed to data samples created by annotators we identify.
\n< p>Biography\nMark Yatskar is an Assistan t Professor at University of Pennsylvania in the department of Computer an d Information Science. He did his PhD at University of Washington co-advis ed by Luke Zettlemoyer and Ali Farhadi. He was a Young Investigator at the Allen Institute for Artificial Intelligence for several years working wit h their computer vision team\, Prior. His work spans Natural Language Proc essing\, Computer Vision\, and Fairness in Machine Learning. He received a Best Paper Award at EMNLP for work on gender bias amplification\, and his work has been featured in Wired and the New York Times.
\n\n X-TAGS;LANGUAGE=en-US:2023\,February\,Yatskar END:VEVENT BEGIN:VEVENT UID:ai1ec-23314@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nWhile GPT models have shown impressive performance on summarization and open-ended text generation\, it’s important to assess t heir abilities on more constrained text generation tasks that require sign ificant and diverse rewritings. In this talk\, I will discuss the challeng es of evaluating systems that are highly competitive and perform close to humans on two such tasks: (i) paraphrase generation and (ii) text simplifi cation. To address these challenges\, we introduce an interactive Rank-and -Rate evaluation framework. Our results show that GPT-3.5 has made a major step up from fine-tuned T5 in paraphrase generation\, but still lacks the diversity and creativity of humans who spontaneously produce large quanti ties of paraphrases.\nAdditionally\, we demonstrate that GPT-3.5 performs similarly to a single human in text simplification\, which makes it diffic ult for existing automatic evaluation metrics to distinguish between the t wo. To overcome this shortcoming\, we propose LENS\, a learnable evaluatio n metric that outperforms SARI\, BERTScore\, and other existing methods in both automatic evaluation and minimal risk decoding for text generation. \nBiography\nWei Xu is an assistant professor in the School of Interactive Computing at the Georgia Institute of Technology\, where she is also affi liated with the new NSF AI CARING Institute and Machine Learning Center. S he received her Ph.D. in Computer Science from New York University and her B.S. and M.S. from Tsinghua University. Xu’s research interests are in na tural language processing\, machine learning\, and social media\, with a f ocus on text generation\, stylistics\, robustness and controllability of m achine learning models\, and reading and writing assistive technology. She is a recipient of the NSF CAREER Award\, CrowdFlower AI for Everyone Awar d\, Criteo Faculty Research Award\, and Best Paper Award at COLING’18. She has also received funds from DARPA and IARPA. She is an elected member of the NAACL executive board and regularly serves as a senior area chair for AI/NLP conferences. DTSTART;TZID=America/New_York:20230224T120000 DTEND;TZID=America/New_York:20230224T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Wei Xu (Georgia Tech) “GPT-3 vs Humans: Rethinking Evaluation of Na tural Language Generation” URL:https://www.clsp.jhu.edu/events/wei-xu-georgia-tech/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n
\\nAbstr act
\nWhile GPT mo dels have shown impressive performance on summarization and open-ended tex t generation\, it’s important to assess their abilities on more constraine d text generation tasks that require significant and diverse rewritings. I n this talk\, I will discuss the challenges of evaluating systems that are highly competitive and perform close to humans on two such tasks: (i) par aphrase generation and (ii) text simplification. To address these challeng es\, we introduce an interactive Rank-and-Rate evaluation framework. Our r esults show that GPT-3.5 has made a major step up from fine-tuned T5 in pa raphrase generation\, but still lacks the diversity and creativity of huma ns who spontaneously produce large quantities of paraphrases.
\nAdditionally\, we demon strate that GPT-3.5 performs similarly to a single human in text simplific ation\, which makes it difficult for existing automatic evaluation metrics to distinguish between the two. To overcome this shortcoming\, we propose LENS\, a learnable evaluation metric that outperforms SARI\, BERTScore\, and other existing methods in both automatic evaluation and minimal risk d ecoding for text generation.
\nBiography
\nWei Xu is an assis tant professor in the School of Interactive Computing at the Georgia Insti tute of Technology\, where she is also affiliated with the new NSF AI CARI NG Institute and Machine Learning Center. She received her Ph.D. in Comput er Science from New York University and her B.S. and M.S. from Tsinghua Un iversity. Xu’s research interests are in natural language processing\, mac hine learning\, and social media\, with a focus on text generation\, styli stics\, robustness and controllability of machine learning models\, and re ading and writing assistive technology. She is a recipient of the NSF CARE ER Award\, CrowdFlower AI for Everyone Award\, Criteo Faculty Research Awa rd\, and Best Paper Award at COLING’18. She has also received funds from D ARPA and IARPA. She is an elected member of the NAACL executive board and regularly serves as a senior area chair for AI/NLP conferences.
\n X-TAGS;LANGUAGE=en-US:2023\,February\,Xu END:VEVENT BEGIN:VEVENT UID:ai1ec-23316@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nUnderstanding the implications underlying a text is c ritical to assessing its impact\, in particular the social dynamics that m ay result from a reading of the text. This requires endowing artificial in telligence (AI) systems with pragmatic reasoning\, for example to correctl y conclude that the statement “Epidemics and cases of disease in the 21st century are “staged”” relates to unfounded conspiracy theories. In this ta lk\, I discuss how shortcomings in the ability of current AI systems to re ason about pragmatics present challenges to equitable detection of false o r harmful language. I demonstrate how these shortcomings can be addressed by imposing human-interpretable structure on deep learning architectures u sing insights from linguistics.In the first part of the talk\, I describe how adversarial text generation algorithms can be used to improve robustne ss of content moderation systems. I then introduce a pragmatic formalism f or reasoning about harmful implications conveyed by social media text. I s how how this pragmatic approach can be combined with generative neural lan guage models to uncover implications of news headlines. I also address the bottleneck to progress in text generation posed by gaps in evaluation of factuality. I conclude by showing how context-aware content moderation can be used to ensure safe interactions with conversational agents.\n \nBiogr aphy\nSaadia Gabriel is a PhD candidate in the Paul G. Allen School of Com puter Science & Engineering at the University of Washington\, advised by P rof. Yejin Choi and Prof. Franziska Roesner. Her researchrevolves around n atural language processing and machine learning\, with a particular focus on building systems for understanding how social commonsense manifests in text (i.e. how do people typically behave in social scenarios)\, as well a s mitigating spread of false or harmful text (e.g. Covid-19 misinformation ). Her work has been covered by a wide range of media outlets like Forbes and TechCrunch. It has also received a 2019 ACL best short paper nominatio n\, a 2019 IROS RoboCup best paper nomination and won a best paper award a t the 2020 WeCNLP summit. Prior to her PhD\, Saadia received a BA summa cu m laude from Mount Holyoke College in Computer Science and Mathematics.\n DTSTART;TZID=America/New_York:20230227T120000 DTEND;TZID=America/New_York:20230227T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Saadia Gabriel (University of Washington) “Socially Responsible and Factual Reasoning for Equitable AI Systems” URL:https://www.clsp.jhu.edu/events/saadia-gabriel-university-of-washington -socially-responsible-and-factual-reasoning-for-equitable-ai-systems/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nUnderstanding the implications underlying a text is c
ritical to assessing its impact\, in particular the social dynamics that m
ay result from a reading of the text. This requires endowing artificial in
telligence (AI) systems with pragmatic reasoning\, for example to correctl
y conclude that the statement “Epidemics and cases of disease in the 21st
century are “staged”” relates to unfounded conspiracy theories. In this ta
lk\, I discuss how shortcomings in the ability of current AI systems to re
ason about pragmatics present challenges to equitable detection of false o
r harmful language. I demonstrate how these shortcomings can be addressed
by imposing human-interpretable structure on deep learning architectures u
sing insights from linguistics.
In the first part of the talk\, I describe how adversarial text gen
eration algorithms can be used to improve robustness of content moderation
systems. I then introduce a pragmatic formalism for reasoning about harmf
ul implications conveyed by social media text. I show how this pragmatic a
pproach can be combined with generative neural language models to uncover
implications of news headlines. I also address the bottleneck to progress
in text generation posed by gaps in evaluation of factuality. I conclude b
y showing how context-aware content moderation can be used to ensure safe
interactions with conversational agents.
\n
Biography
\nSaadia Gabr iel is a PhD candidate in the Paul G. Allen School of Computer Scie nce & Engineering at the University of Washington\, advised by Prof. Yejin Choi and Prof. Franziska Roesner. Her research re volves around natural language processing and machine learning\, with a pa rticular focus on building systems for understanding how social commonsens e manifests in text (i.e. how do people typically behave in social scenari os)\, as well as mitigating spread of false or harmful text (e.g. Covid-19 misinformation). Her work has been covered by a wide range of media outle ts like Forbes and TechCrunch. It has also received a 2019 ACL best short paper nomination\, a 2019 IROS RoboCup best paper nomination and won a bes t paper award at the 2020 WeCNLP summit. Prior to her PhD\, Saadia received a BA summa cum laude from Mount Holyoke College in Computer Sc ience and Mathematics.
\n\n X-TAGS;LANGUAGE=en-US:2023\,February\,Gabriel END:VEVENT BEGIN:VEVENT UID:ai1ec-23312@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nAdvanced neural language models have grown ever large r and more complex\, pushing forward the limits of language understanding and generation\, while diminishing interpretability. The black-box nature of deep neural networks blocks humans from understanding them\, as well as trusting and using them in real-world applications. This talk will introd uce interpretation techniques that bridge the gap between humans and model s for developing trustworthy natural language processing(NLP). I will firs t show how to explain black-box models and evaluate their explanations for understanding their prediction behavior. Then I will introduce how to imp rove the interpretability of neural language models by making their decisi on-making transparent and rationalized. Finally\, I will discuss how to di agnose and improve models (e.g.\, robustness) through the lens of explanat ions. I will conclude with future research directions that are centered ar ound model interpretability and committed to facilitating communications a nd interactions between intelligent machines\, system developers\, and end users for long-term trustworthy AI.\nBiography\nHanjie Chen is a Ph.D. ca ndidate in Computer Science at the University of Virginia\, advised by Pro f. Yangfeng Ji. Her research interests lie in Trustworthy AI\, Natural Lan guage Processing (NLP)\, andInterpretable Machine Learning. She develops i nterpretation techniques to explain neural language models and make their prediction behavior transparent and reliable. She is a recipient of the Ca rlos and Esther Farrar Fellowship and the Best Poster Award at the ACM CAP WIC 2021. Her work has been published at top-tier NLP/AI conferences (e.g. \, ACL\, AAAI\, EMNLP\, NAACL) and selected by the National Center for Wom en & Information Technology (NCWIT) Collegiate Award Finalist 2021. She (a s the primary instructor) co-designed and taught the course\, Interpretabl e Machine Learning\, and was awarded the UVA CS Outstanding Graduate Teach ing Award and University-wide Graduate Teaching Awards Nominee (top 5% of graduate instructors). More details can be found athttps://www.cs.virginia .edu/~hc9mx DTSTART;TZID=America/New_York:20230313T120000 DTEND;TZID=America/New_York:20230313T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Hanjie Chen (University of Virginia) “Bridging Humans and Machines: Techniques for Trustworthy NLP” URL:https://www.clsp.jhu.edu/events/hanjie-chen-university-of-virginia/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\n
Abstr act
\nAdvanced neural language models have grown ever large
r and more complex\, pushing forward the limits of language understanding
and generation\, while diminishing interpretability. The black-box nature
of deep neural networks blocks humans from understanding them\, as well as
trusting and using them in real-world applications. This talk will introd
uce interpretation techniques that bridge the gap between humans and model
s for developing trustworthy natural language processing
(NLP). I will first show how to explain black-box models and evalua
te their explanations for understanding their prediction behavior. Then I
will introduce how to improve the interpretability of neural language mode
ls by making their decision-making transparent and rationalized. Finally\,
I will discuss how to diagnose and improve models (e.g.\, robustness) thr
ough the lens of explanations. I will conclude with future research direct
ions that are centered around model interpretability and committed to faci
litating communications and interactions between intelligent machines\, sy
stem developers\, and end users for long-term trustworthy AI.
Hanjie Chen is a Ph.D. candidate in Compute r Science at the University of Virginia\, advised by Prof. Yangfeng Ji. He r research interests lie in Trustworthy AI\, Natural Language Processing ( NLP)\, and
\n X-TAGS;LANGUAGE=en-US:2023\,Chen\,February END:VEVENT BEGIN:VEVENT UID:ai1ec-23515@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION:Abstract\n\n\n\nHow important are different temporal speech mod ulations for speech recognition? We answer this question from two compleme ntary perspectives. Firstly\, we quantify the amount of phonetic informati on in the modulation spectrum of speech by computing the mutual informatio n between temporal modulations with frame-wise phoneme labels. Looking fro m another perspective\, we ask – which speech modulations an Automatic Spe ech Recognition (ASR) system prefers for its operation. Data-driven weight s are learned over the modulation spectrum and optimized for an end-to-end ASR task. Both methods unanimously agree that speech information is mostl y contained in slow modulation. Maximum mutual information occurs around 3 -6 Hz which also happens to be the range of modulations most preferred by the ASR. In addition\, we show that the incorporation of this knowledge in to ASRs significantly reduces their dependency on the amount of training d ata.\n DTSTART;TZID=America/New_York:20230403T120000 DTEND;TZID=America/New_York:20230403T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Student Seminar – Samik Sadhu (JHU) “Importance of Different Tempor al Modulations of Speech: A Tale of Two Perspectives” URL:https://www.clsp.jhu.edu/events/student-seminar-samik-sadhu/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\n Interpretable Machine Learning. She dev elops interpretation techniques to explain neural language models and make their prediction behavior transparent and reliable. She is a recipient of the Carlos and Esther Farrar Fellowship and the Best Poster Award at the ACM CAPWIC 2021. Her work has been published at top-tier NLP/AI conference s (e.g.\, ACL\, AAAI\, EMNLP\, NAACL) and selected by the National Center for Women & Information Technology (NCWIT) Collegiate Award Finalist 2021. She (as the primary instructor) co-designed and taught the course\, Inter pretable Machine Learning\, and was awarded the UVA CS Outstanding Graduat e Teaching Award and University-wide Graduate Teaching Awards Nominee (top 5% of graduate instructors). More details can be found at https://www.cs.virginia.edu/~hc9mxAbstr act
\nHow important are different temporal speech modulations for speec h recognition? We answer this question from two complementary perspectives . Firstly\, we quantify the amount of phonetic information in the modulati on spectrum of speech by computing the mutual information between temporal modulations with frame-wise phoneme labels. Looking from another perspect ive\, we ask – which speech modulations an Automatic Speech Recognition (A SR) system prefers for its operation. Data-driven weights are learned over the modulation spectrum and optimized for an end-to-end ASR task. Both me thods unanimously agree that speech information is mostly contained in slo w modulation. Maximum mutual information occurs around 3-6 Hz which also h appens to be the range of modulations most preferred by the ASR. In additi on\, we show that the incorporation of this knowledge into ASRs significan tly reduces their dependency on the amount of training data.
\n< p> \nLearning How to Play With The Machines: Taking Stock of Where the Collaboration Between Computational and Social Science Stands
\n\n
Speakers: Jeff Gill\, Ernesto Calvo\, Hale Sirin and Antonios Anastasopoulos
\n X-TAGS;LANGUAGE=en-US:2023\,April\,APSA Roundtable END:VEVENT BEGIN:VEVENT UID:ai1ec-23586@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION: DTSTART;TZID=America/New_York:20230410T120000 DTEND;TZID=America/New_York:20230410T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Student Seminar – Ruizhe Huang URL:https://www.clsp.jhu.edu/events/student-seminar-ruizhe-huang/ X-COST-TYPE:free X-TAGS;LANGUAGE=en-US:2023\,April\,Huang END:VEVENT BEGIN:VEVENT UID:ai1ec-23588@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nAdvances in open domain Large Language Models (LLMs) starting with BERT and more recently with GPT-4\, PaLM\, and LLaMA have fa cilitated dramatic improvements in conversational systems. These improveme nts include an unprecedented breadth of conversational interactions betwee n humans and machines while maintaining and sometimes surpassing the accur acy of systems trained specifically for known\, closed domains. However\, many applications still require higher levels of accuracy than pre-trained LLMs can provide. There are many studies underway to accomplish this. Bro adly speaking\, the methods assume the pre-trained models are fixed (due t o cost/time)\, and instead look to various augmentation methods including prompting strategies and model adaptation/fine-tuning.\nOne augmentation s trategy leverages the context of the conversation. For example\, who are t he participants and what is known about these individuals (personal contex t)\, what was just said (dialogue context)\, where is the conversation tak ing place (geo context)\, what time of day and season is it (time context) \, etc. A powerful form of context is the shared visual setting of the co nversation between the human(s) and machine. The shared visual scene may b e from a device (phone\, smart glasses) or represented on a screen (browse r\, maps\, etc.) The elements in the visual context can be exploited by gr ounding the natural language conversational interaction\, thereby changing the priors of certain concepts and increasing the accuracy of the system. In this talk\, I will present some of my historical work in this area as well as my recent work in the AI Virtual Assistant (AVA) Lab at Georgia Te ch.\nBio\nDr. Larry Heck is a Professor with a joint appointment in the Sc hool of Electrical and Computer Engineering and the School of Interactive Computing at the Georgia Institute of Technology. He holds the Rhesa S. Fa rmer Distinguished Chair of Advanced Computing Concepts and is a Georgia R esearch Alliance Eminent Scholar. His received the BSEE from Texas Tech Un iversity (1986)\, and MSEE and PhD EE from the Georgia Institute of Techno logy (1989\,1991). He is a Fellow of the IEEE\, inducted into the Academy of Distinguished Engineering Alumni at Georgia Tech and received the Disti nguished Engineer Award from the Texas Tech University Whitacre College of Engineering. He was a Senior Research Engineer with SRI (1992-98)\, Vice President of R&D at Nuance (1998-2005)\, Vice President of Search and Adve rtising Sciences at Yahoo! (2005-2009)\, Chief Scientist of the Microsoft Speech products and Distinguished Engineer in Microsoft Research (2009-201 4)\, Principal Scientist with Google Research (2014-2017)\, and CEO of Viv Labs and SVP at Samsung (2017-2021).\n\n DTSTART;TZID=America/New_York:20230414T120000 DTEND;TZID=America/New_York:20230414T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Larry Heck (Georgia Institute of Technology) “The AVA Digital Human : Improving Conversational Interactions through Visually Situated Context” URL:https://www.clsp.jhu.edu/events/larry-heck-georgia-institute-of-technol ogy-the-ava-digital-human-improving-conversational-interactions-through-vi sually-situated-context/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nAdvances in open domain Large Lan guage Models (LLMs) starting with BERT and more recently with GPT-4\, PaLM \, and LLaMA have facilitated dramatic improvements in conversational syst ems. These improvements include an unprecedented breadth of conversational interactions between humans and machines while maintaining and sometimes surpassing the accuracy of systems trained specifically for known\, closed domains. However\, many applications still require higher levels of accur acy than pre-trained LLMs can provide. There are many studies underway to accomplish this. Broadly speaking\, the methods assume the pre-trained mod els are fixed (due to cost/time)\, and instead look to various augmentatio n methods including prompting strategies and model adaptation/fine-tuning.
\nOne augmentation strategy leverages the conte xt of the conversation. For example\, who are the participants and what is known about these individuals (personal context)\, what was just said (di alogue context)\, where is the conversation taking place (geo context)\, w hat time of day and season is it (time context)\, etc. A powerful form of context is the shared visual setting of the conversation between the huma n(s) and machine. The shared visual scene may be from a device (phone\, sm art glasses) or represented on a screen (browser\, maps\, etc.) The elemen ts in the visual context can be exploited by grounding the natural languag e conversational interaction\, thereby changing the priors of certain conc epts and increasing the accuracy of the system. In this talk\, I will pres ent some of my historical work in this area as well as my recent work in t he AI Virtual Assistant (AVA) Lab at Georgia Tech.
\nBio
\nDr. Larry Heck is a Professor with a joi nt appointment in the School of Electrical and Computer Engineering and th e School of Interactive Computing at the Georgia Institute of Technology. He holds the Rhesa S. Farmer Distinguished Chair of Advanced Computing Con cepts and is a Georgia Research Alliance Eminent Scholar. His received the BSEE from Texas Tech University (1986)\, and MSEE and PhD EE from the Geo rgia Institute of Technology (1989\,1991). He is a Fellow of the IEEE\, in ducted into the Academy of Distinguished Engineering Alumni at Georgia Tec h and received the Distinguished Engineer Award from the Texas Tech Univer sity Whitacre College of Engineering. He was a Senior Research Engineer wi th SRI (1992-98)\, Vice President of R&D at Nuance (1998-2005)\, Vice Pres ident of Search and Advertising Sciences at Yahoo! (2005-2009)\, Chief Sci entist of the Microsoft Speech products and Distinguished Engineer in Micr osoft Research (2009-2014)\, Principal Scientist with Google Research (201 4-2017)\, and CEO of Viv Labs and SVP at Samsung (2017-2021).
\n\n
\n X-TAGS;LANGUAGE=en-US:2023\,April\,Heck END:VEVENT BEGIN:VEVENT UID:ai1ec-23590@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nMachine Translation has the ultimate goal of eliminat ing language barriers. However\, the area has focused mainly on a few lang uages\, leaving many low-resource languages without support. In this talk\ , I will discuss the challenges of bringing translation support for 200 wr itten languages and beyond.\n\nFirst\, I talk about the No Language Left B ehind Project\, where we took on this challenge by first contextualizing t he need for low-resource language translation support through exploratory interviews with native speakers. Then\, we created datasets and models aim ed at narrowing the performance gap between low and high-resource language s. We proposed multiple architectural and training improvements to counter act over-fitting while training on thousands of language-pairs/tasks. We e valuated the performance of over 40\,000 different translation directions. \n\nAfterwards\, I’ll discuss the challenges of pushing translation perfor mance beyond text for languages that don’t have written standards like Hok kien.\nOur models achieve state-of-the-art performance and lay important g roundwork towards realizing a universal translation system. At the same ti me\, we keep making open-source contributions for everyone to keep advanci ng the research for the languages they care about.\nBio\nPaco is Research Scientist Manager supporting translation teams in Meta AI (FAIR). He works in the field of machine translation with a focus on low-resource translat ion (e.g. NLLB\, FLORES) and the aim to break language barriers. He joined Meta in 2016. His research has been published in top-tier NLP venues like ACL\, EMNLP. He was the co-chair of the Research director at AMTA (2020-2 022). He has ave organized several research competitions focused on low-re source translation and data filtering. Paco obtained his PhD from the ITES M in Mexico\, was a visiting scholar at the LTI-CMU from 2008-2009\, and p articipated in DARPA’s GALE evaluation program. Paco was a post-doc and sc ientist at Qatar Computing Research Institute in Qatar in 2012-2016 DTSTART;TZID=America/New_York:20230417T120000 DTEND;TZID=America/New_York:20230417T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Paco Guzman (Meta AI) “Building a Universal Translation System to B reak Down Language Barriers” URL:https://www.clsp.jhu.edu/events/paco-guzman-meta-ai/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n
\\nAbstr act
\nOur models achieve state-of-the-art performance and lay important groundwork towards realizing a universal translation system. At the same time\, we keep maki ng open-source contributions for everyone to keep advancing the research f or the languages they care about.
\nBio
\nPac o is Research Scientist Manager supporting translation teams in Meta AI (F AIR). He works in the field of machine translation with a focus on low-res ource translation (e.g. NLLB\, FLORES) and the aim to break language barri ers. He joined Meta in 2016. His research has been published in top-tier N LP venues like ACL\, EMNLP. He was the co-chair of the Research director a t AMTA (2020-2022). He has ave organized several research competitions foc used on low-resource translation and data filtering. Paco obtained his PhD from the ITESM in Mexico\, was a visiting scholar at the LTI-CMU from 200 8-2009\, and participated in DARPA’s GALE evaluation program. Paco was a p ost-doc and scientist at Qatar Computing Research Institute in Qatar in 20 12-2016
\n X-TAGS;LANGUAGE=en-US:2023\,April\,Guzman END:VEVENT BEGIN:VEVENT UID:ai1ec-23592@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nLarge language models (LLMs) have ushered in exciting capabilities in language understanding and text generation\, with systems like ChatGPT holding fluent dialogs with users and being almost indisting uishable from humans. While this has obviously raised conversational syste ms and chatbots to a new level\, it also presents exciting new opportuniti es for building artificial agents with improved decision making capabiliti es. Specifically\, the ability to reason with language can allow us to bui ld agents that can 1) execute complex action sequences to effect change in the world\, 2) learn new skills by ‘reading’ in addition to ‘doing’\, and 3) allow for easier personalization and control over their behavior. In t his talk\, I will demonstrate how we can build such language-enabled agent s that exhibit the above traits across various use cases such as multi-hop question answering\, web interaction\, and robotic tool manipulation. In the end\, I will also discuss some dangers of using these LLM-based system s and some challenges that lie ahead in ensuring their safe use.\nBiograph y\nKarthik Narasimhan is an assistant professor in the Computer Science de partment at Princeton University and a co-Director of the Princeton NLP gr oup. His research spans the areas of natural language processing and reinf orcement learning\, with the goal of building intelligent agents that lear n to operate in the world through both their own experience (”doing things ”) and leveraging existing human knowledge (”reading about things”). Karth ik received his PhD from MIT in 2017\, and spent a year as a visiting rese arch scientist at OpenAI contributing to the GPT language model\, prior to joining Princeton in 2018. His research has been recognized by the NSF CA REER\, a Google Research Scholar Award\, an Amazon research award (2019)\, Bell Labs runner-up prize and outstanding paper awards at EMNLP (2015\, 2 016) and NeurIPS (2022). DTSTART;TZID=America/New_York:20230421T120000 DTEND;TZID=America/New_York:20230421T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Karthik Narasimhan (Princeton University) ” Towards General-Purpose Language-Enabled Agents: Machines that can Read\, Think and Act” URL:https://www.clsp.jhu.edu/events/karthik-narasimhan-princeton-university / X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nLarge language models (LLMs) have ushered in exciting capabilities in language understanding and text generation\, with systems like ChatGPT holding fluent dialogs with users and being almost indisting uishable from humans. While this has obviously raised conversational syste ms and chatbots to a new level\, it also presents exciting new opportuniti es for building artificial agents with improved decision making capabiliti es. Specifically\, the ability to reason with language can allow us to bui ld agents that can 1) execute complex action sequences to effect change in the world\, 2) learn new skills by ‘reading’ in addition to ‘doing’\, and 3) allow for easier personalization and control over their behavior. In t his talk\, I will demonstrate how we can build such language-enabled agent s that exhibit the above traits across various use cases such as multi-hop question answering\, web interaction\, and robotic tool manipulation. In the end\, I will also discuss some dangers of using these LLM-based system s and some challenges that lie ahead in ensuring their safe use.
\n< strong>Biography
\nKarthik Narasimhan is an assistan t professor in the Computer Science department at Princeton University and a co-Director of the Princeton NLP group. His research spans the areas of natural language processing and reinforcement learning\, with the goal of building intelligent agents that learn to operate in the world through bo th their own experience (”doing things”) and leveraging existing human kno wledge (”reading about things”). Karthik received his PhD from MIT in 2017 \, and spent a year as a visiting research scientist at OpenAI contributin g to the GPT language model\, prior to joining Princeton in 2018. His rese arch has been recognized by the NSF CAREER\, a Google Research Scholar Awa rd\, an Amazon research award (2019)\, Bell Labs runner-up prize and outst anding paper awards at EMNLP (2015\, 2016) and NeurIPS (2022).
\n X-TAGS;LANGUAGE=en-US:2023\,April\,Narasimhan END:VEVENT BEGIN:VEVENT UID:ai1ec-23606@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION: DTSTART;TZID=America/New_York:20230424T120000 DTEND;TZID=America/New_York:20230424T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Student Seminar – Brian Lu URL:https://www.clsp.jhu.edu/events/student-seminar-brian-lu/ X-COST-TYPE:free X-TAGS;LANGUAGE=en-US:2023\,April\,Lu END:VEVENT BEGIN:VEVENT UID:ai1ec-23608@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nAutomated analysis of student writing has the potenti al to provide alternatives to selected-response questions such as multiple choice\, and to enable teachers and instructors to assess students’ reaso ning skills based on their long-form writing. Further\, automated support to assess both short answers and long passages could provide students with a smoother trajectory towards mastery of written communication. Our meth ods focus on the specific ideas students express to support formative asse ssment through different kinds of feedback\, which aims to scaffold their abilities to reason and communicate. In this talk I review our work in the PSU NLP lab on methods for automated assessment of different forms of stu dent writing\, from younger and older students. I will briefly illustrate highly curated datasets created in collaboration with researchers in STEM education\, results from deployment of an older content analysis tool on middle school physics essays\, and very preliminary results on assessment of college students’ physics lab reports. I will also present our current work on short answer assessment using a novel recurrent relation network that incorporates contrastive learning.\nBio\nBecky Passonneau has been a Professor in the Department of Computer Science and Engineering at Penn St ate University since 2016\, when she joined as the first NLP researcher. S ince that time the NLP faculty has grown to include Rui Zhang and Wenpeng Yin. Becky’s research in natural language processing addresses computation al pragmatics\, meaning the investigation of language as a system of inter active behavior that serves a wide range of purposes. She received her PhD in Linguistics from the University of Chicago in 1985\, and worked at sev eral academic and industry research labs before joining Penn State. Her wo rk is reported in over 140 publications in journals and refereed conferenc e proceedings\, and has been funded through 27 sponsored projects from 16 sources\, including government agencies\, corporate sponsors\, corporate gifts\, and foundations.. DTSTART;TZID=America/New_York:20230428T120000 DTEND;TZID=America/New_York:20230428T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Becky Passonneau (Penn State University) ” Automated Support to Sca ffold Students’ Short- and Long-form STEM Writing” URL:https://www.clsp.jhu.edu/events/becky-passonneau-penn-state-university- automated-support-to-scaffold-students-short-and-long-form-stem-writing/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nAutomated analysis of student writing has the potenti al to provide alternatives to selected-response questions such as multiple choice\, and to enable teachers and instructors to assess students’ reaso ning skills based on their long-form writing. Further\, automated support to assess both short answers and long passages could provide students with a smoother trajectory towards mastery of written communication. Our meth ods focus on the specific ideas students express to support formative asse ssment through different kinds of feedback\, which aims to scaffold their abilities to reason and communicate. In this talk I review our work in the PSU NLP lab on methods for automated assessment of different forms of stu dent writing\, from younger and older students. I will briefly illustrate highly curated datasets created in collaboration with researchers in STEM education\, results from deployment of an older content analysis tool on middle school physics essays\, and very preliminary results on assessment of college students’ physics lab reports. I will also present our current work on short answer assessment using a novel recurrent relation network that incorporates contrastive learning.
\nBio
\nBecky Passonneau has been a Professor in the Department of Computer Sci ence and Engineering at Penn State University since 2016\, when she joined as the first NLP researcher. Since that time the NLP faculty has grown to include Rui Zhang and Wenpeng Yin. Becky’s research in natural language p rocessing addresses computational pragmatics\, meaning the investigation o f language as a system of interactive behavior that serves a wide range of purposes. She received her PhD in Linguistics from the University of Chic ago in 1985\, and worked at several academic and industry research labs be fore joining Penn State. Her work is reported in over 140 publications in journals and refereed conference proceedings\, and has been funded through 27 sponsored projects from 16 sources\, including government agencies\, corporate sponsors\, corporate gifts\, and foundations..
\n X-TAGS;LANGUAGE=en-US:2023\,April\,Passonneau END:VEVENT BEGIN:VEVENT UID:ai1ec-24241@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nOur research focuses on improving speech processing a lgorithms\, such as automatic speech recognition (ASR)\, speaker identific ation\, and depression detection\, under challenging conditions such as li mited data (for example\, children’s or clinical speech)\, mismatched cond itions (for example\, training on read speech while recognizing conversati onal speech)\, and noisy speech\, using a hybrid data-driven and knowledge -based approach. This approach requires understanding of both machine lear ning approaches and of the human speech production and perception systems. I will summarize in this talk our work on children’s ASR using self-super vised models\, detecting depression from speech signals using novel speake r disentaglement techniques\, and automating scoring of children’s reading tasks with both ASR and innovative NLP algorithms.\nBiography\nAbeer Alwa n received her Ph.D. in Electrical Engineering and Computer Science from M IT in 1992. Since then\, she has been with the ECE department at UCLA wher e she is now a Full Professor and directs the Speech Processing and Audito ry Perception Laboratory. She is the recipient of the NSF Research Initiat ion and Career Awards\, NIH FIRST Award\, UCLA-TRW Excellence in Teaching Award\, Okawa Foundation Award in Telecommunication\, and the Engineer’s C ouncil Educator Award. She is a Fellow of the Acoustical Society of Americ a\, IEEE\, and International Speech Communication Assoc. (ISCA). She was a Fellow at the Radcliffe Institute\, Harvard University\, co-Editor in Chi ef of Speech Communication\, Associate Editor of JASA and IEEE TSALP\, a D istinguished Lecturer of ISCA\, a member of the IEEE Signal Processing Boa rd of Governers and she is currently on the advisory board of ISCA and the UCLA-Amazon Science Hub for Humanity and AI. DTSTART;TZID=America/New_York:20240202T120000 DTEND;TZID=America/New_York:20240202T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Abeer Alwan (UCLA) “Dealing with Limited Speech Data and Variabilit y: Three case studies” URL:https://www.clsp.jhu.edu/events/abeer-alwan-ucla/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nOur research focuses on improving speech processing a lgorithms\, such as automatic speech recognition (ASR)\, speaker identific ation\, and depression detection\, under challenging conditions such as li mited data (for example\, children’s or clinical speech)\, mismatched cond itions (for example\, training on read speech while recognizing conversati onal speech)\, and noisy speech\, using a hybrid data-driven and knowledge -based approach. This approach requires understanding of both machine lear ning approaches and of the human speech production and perception systems. I will summarize in this talk our work on children’s ASR using self-super vised models\, detecting depression from speech signals using novel speake r disentaglement techniques\, and automating scoring of children’s reading tasks with both ASR and innovative NLP algorithms.
\nBiogra phy
\nAbeer Alwan received her Ph.D. in Electrical Engineer ing and Computer Science from MIT in 1992. Since then\, she has been with the ECE department at UCLA where she is now a Full Professor and directs t he Speech Processing and Auditory Perception Laboratory. She is the recipi ent of the NSF Research Initiation and Career Awards\, NIH FIRST Award\, U CLA-TRW Excellence in Teaching Award\, Okawa Foundation Award in Telecommu nication\, and the Engineer’s Council Educator Award. She is a Fellow of t he Acoustical Society of America\, IEEE\, and International Speech Communi cation Assoc. (ISCA). She was a Fellow at the Radcliffe Institute\, Harvar d University\, co-Editor in Chief of Speech Communication\, Associate Edit or of JASA and IEEE TSALP\, a Distinguished Lecturer of ISCA\, a member of the IEEE Signal Processing Board of Governers and she is currently on the advisory board of ISCA and the UCLA-Amazon Science Hub for Humanity and A I.
\n X-TAGS;LANGUAGE=en-US:2024\,Alwan\,February END:VEVENT BEGIN:VEVENT UID:ai1ec-24425@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION:Abstract\n\nOver the past three decades\, the fields of automat ic speech recognition (ASR) and machine translation (MT) have witnessed re markable advancements\, leading to exciting research directions such as sp eech-to-text translation (ST). This talk will delve into the domain of con versational ST\, an essential facet of daily communication\, which present s unique challenges including spontaneous informal language\, the presence of disfluencies\, high context dependence and a scarcity of ST paired dat a.\n\nConversational speech is notably characterized by its reliance on sh ort segments\, requiring the integration of broader contexts to maintain c onsistency and improve the translation’s fluency and quality. Incorporati ng longer contexts has been shown to benefit machine translation\, but the inclusion of context in E2E-ST remains under-studied. Previous approaches have used simple concatenation of audio inputs for context\, leading to m emory bottlenecks\, especially in self-attention networks\, due to the enc oding of lengthy audio segments.\n\nFirst\, I will describe how to integra te the context into E2E-ST with minimum additional memory cost. Then\, I will discuss the challenges of incorporating context in an E2E-ST system w ith limited data during training and inference and propose solutions to ov ercome them. Afterward\, I will illustrate the impact of context size and the inclusion of speaker information on performance. Lastly\, I will demon strate the benefits of context in conversational settings focusing on asp ects like anaphora resolution and the identification of named entities. DTSTART;TZID=America/New_York:20240205T120000 DTEND;TZID=America/New_York:20240205T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Amir Hussein “Towards End-to-End Conversational Speech Translation” URL:https://www.clsp.jhu.edu/events/amir-hussein-towards-end-to-end-convers ational-speech-translation/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr
act
\n
Over the past three decades\, the fields of automatic speech recognition (ASR) and machine tra nslation (MT) have witnessed remarkable advancements\, leading to exciting research directions such as speech-to-text translation (ST). This talk wi ll delve into the domain of conversational ST\, an essential facet of dail y communication\, which presents unique challenges including spontaneous i nformal language\, the presence of disfluencies\, high context dependence and a scarcity of ST paired data.
\n\nAbstr act
\nThere is an enormous data gap between how AI systems and children learn language: The best LLMs now learn language from text with a word count in the trillions\, whereas it would take a child roughly 100K years to reach those numbers through speec h (Frank\, 2023\, “Bridging the data gap”). There is also a clear generali zation gap: whereas machines struggle with systematic generalization\, peo ple excel. For instance\, once a child learns how to “skip\,” they immedia tely know how to “skip twice” or “skip around the room with their hands up ” due to their compositional skills. In this talk\, I’ll describe two case studies in addressing these gaps:
\n1) The dat a gap: We train deep neural networks from scratch (using DINO\, CLIP\, etc .)\, not on large-scale data from the web\, but through the eyes and ears of a single child. Using head-mounted video recordings from a child (61 ho urs of video slices over 19 months)\, we show how deep neural networks can acquire many word-referent mappings\, generalize to novel visual referent s\, and achieve multi-modal alignment. Our results demonstrate how today’s AI models are capable of learning key aspects of children’s early knowled ge from realistic input.
\n2) The generalizatio n gap: Can neural networks capture human-like systematic generalization? W e address a 35-year-old debate catalyzed by Fodor and Pylyshyn’s classic a rticle\, which argued that standard neural networks are not viable models of the mind because they lack systematic compositionality — the algebraic ability to understand and produce novel combinations from known components . We’ll show how neural network can achieve human-like systematic generali zation when trained through meta-learning for compositionality (MLC)\, a n ew method for optimizing the compositional skills of neural networks throu gh practice. With MLC\, a neural network can match human performance and s olve several machine learning benchmarks.
\nGiv en this work\, we’ll discuss the paths forward for building machines that learn\, generalize\, and interact in more human-like ways based on more na tural input.
\nRelated articles:
\nVong\, W. K.\, Wang\, W.\, Orhan\, A. E.\, and Lake\, B. M (2024). Grounded language acquisition through the eyes and ears of a singl e child. Science\, 383.
\nOrhan\, A. E.\ , and Lake\, B. M. (in press). Learning high-level visual representations from a child’s perspective without strong inductive biases. Nature Mach ine Intelligence.
\nLake\, B. M. and Baroni \, M. (2023). Human-like systematic generalization through a meta-learning neural network. Nature\, 623\, 115-121.
\nBiography< /strong>
\nBrenden M. Lake is an Assistant Prof essor of Psychology and Data Science at New York University. He received h is M.S. and B.S. in Symbolic Systems from Stanford University in 2009\, an d his Ph.D. in Cognitive Science from MIT in 2014. He was a postdoctoral D ata Science Fellow at NYU from 2014-2017. Brenden is a recipient of the Ro bert J. Glushko Prize for Outstanding Doctoral Dissertation in Cognitive S cience\, he is a MIT Technology Review Innovator Under 35\, and his resear ch was selected by Scientific American as one of the 10 most important adv ances of 2016. Brenden’s research focuses on computational problems that a re easier for people than they are for machines\, such as learning new con cepts\, creating new concepts\, learning-to-learn\, and asking questions.< /p>\n
\\nAbstr act
\nLarge language models like ChatGPT have shown extraor dinary abilities for writing. While impressive at first glance\, large lan guage models aren’t perfect and often make mistakes humans would not make. The main architecture behind ChatGPT mostly doesn’t differ from early neu ral networks\, and as a consequence\, carries some of the same limitations . My work revolves around the use of neural networks like ChatGPT mixed wi th symbolic methods from early AI and how these two families of methods ca n combine to create more robust AI. I talk about some of the neurosymbolic methods I used for applications in story generation and understanding — w ith the goal of eventually creating AI that can play Dungeons & Dragons. I also discuss pain points that I found for improving accessible communicat ion and show how large language models can supplement such communication.< /p>\n
Biography
\nAbstr act
\nWe introduce STAR (Stream Transduction with Anchor Re presentations)\, a novel Transformer-based model designed for efficient se quence-to-sequence transduction over streams. STAR dynamically segments in put streams to create compressed anchor representations\, achieving nearly lossless compression (12x) in Automatic Speech Recognition (ASR) and outp erforming existing methods. Moreover\, STAR demonstrates superior segmenta tion and latency-quality trade-offs in simultaneous speech-to-text tasks\, optimizing latency\, memory footprint\, and quality.
\n X-TAGS;LANGUAGE=en-US:2024\,February\,Tan END:VEVENT BEGIN:VEVENT UID:ai1ec-24429@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nI discuss the application of Foundation Models in Ast ronomy through the collaborative efforts of the UniverseTBD consortium wit h a mission to democratize Science for everyone. One of our key objectives is to overcome the limitations of general-purpose Foundation Models\, suc h as producing limited information in specialized fields. To this end\, we have developed the first specialized large language model for Astronomy\, AstroLLaMa-1. This model\, enhanced by exposure to domain-specific litera ture from the NASA Astrophysics Data System and ArXiv\, demonstrates impro ved text completion and embedding capabilities over existent GPT models. I further discuss the potential of LLMs in generating complex scientific hy potheses and extracting meaningful insights from astronomy literature. Our findings\, validated by human experts\, demonstrate the LLM capability in informed scientific critique and uncover intriguing patterns in the embed ding space\, highlighting the potential of LLMs to augment scientific inqu iry. I will also discuss preliminary work with the multi-modal model Astro LLaVA\, which allows us to interact with astronomical images via natural l anguage. Through the work of UniverseTBD\, we aim to explore how artificia l intelligence can assist human intelligence in Astronomy and\, more broad ly\, Science.\nBiography\nIoana Ciucă\, who goes by Jo\, is an interdiscip linary Jubilee Joint Fellow at the Australian National University\, workin g across the School of Computing and the Research School of Astronomy & As trophysics. Before joining ANU\, Jo finished her PhD in Astrophysics at Un iversity College London in the United Kingdom\, where she worked at the in tersection of Astronomy and Machine Learning to understand the formation a nd evolution history of our Galaxy\, the Milky Way. Jo is now focusing on utilizing foundation models that benefit researchers everywhere\, working alongside the UniverseTBD team of more than 30 astronomers\, engineers\, M L practitioners and enthusiasts worldwide.\n DTSTART;TZID=America/New_York:20240223T120000 DTEND;TZID=America/New_York:20240223T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Ioana Ciuca (Australian National University)”A Universe To Be Decid ed: Towards Specialized Foundation Models for Advancing Astronomy” URL:https://www.clsp.jhu.edu/events/ioana-ciuca-australian-national-univers ity/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nI discuss the application of Foundation Models in Ast ronomy through the collaborative efforts of the UniverseTBD consortium wit h a mission to democratize Science for everyone. One of our key objectives is to overcome the limitations of general-purpose Foundation Models\, suc h as producing limited information in specialized fields. To this end\, we have developed the first specialized large language model for Astronomy\, AstroLLaMa-1. This model\, enhanced by exposure to domain-specific litera ture from the NASA Astrophysics Data System and ArXiv\, demonstrates impro ved text completion and embedding capabilities over existent GPT models. I further discuss the potential of LLMs in generating complex scientific hy potheses and extracting meaningful insights from astronomy literature. Our findings\, validated by human experts\, demonstrate the LLM capability in informed scientific critique and uncover intriguing patterns in the embed ding space\, highlighting the potential of LLMs to augment scientific inqu iry. I will also discuss preliminary work with the multi-modal model Astro LLaVA\, which allows us to interact with astronomical images via natural l anguage. Through the work of UniverseTBD\, we aim to explore how artificia l intelligence can assist human intelligence in Astronomy and\, more broad ly\, Science.
\nBiography
\nIoana Ciucă\, who goes by Jo\, is an interdisciplinary Jubilee Joint Fellow at the Australi an National University\, working across the School of Computing and the Re search School of Astronomy & Astrophysics. Before joining ANU\, Jo finishe d her PhD in Astrophysics at University College London in the United Kingd om\, where she worked at the intersection of Astronomy and Machine Learnin g to understand the formation and evolution history of our Galaxy\, the Mi lky Way. Jo is now focusing on utilizing foundation models that benefit re searchers everywhere\, working alongside the UniverseTBD team of more than 30 astronomers\, engineers\, ML practitioners and enthusiasts worldwide.< /p>\n
\n X-TAGS;LANGUAGE=en-US:2024\,Ciuca\,February END:VEVENT BEGIN:VEVENT UID:ai1ec-24457@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION:Abstract\nAs artificial intelligence (AI) continues to rapidly expand into existing healthcare infrastructure – e.g.\, clinical decision support\, administrative tasks\, and public health surveillance – it is pe rhaps more important than ever to reflect on the broader purpose of such s ystems. While much focus has been on the potential for this technology to improve general health outcomes\, there also exists a significant\, but un derstated\, opportunity to use this technology to address health-related d isparities. Accomplishing the latter depends not only on our ability to ef fectively identify addressable areas of systemic inequality and translate them into tasks that are machine learnable\, but also our ability to measu re\, interpret\, and counteract barriers in training data that may inhibit robustness to distribution shift upon deployment (i.e.\, new populations\ , temporal dynamics). In this talk\, we will discuss progress made along b oth of these dimensions. We will begin by providing background on the stat e of AI for promoting health equity. Then\, we will present results from a recent clinical phenotyping project and discuss their implication on prev ailing views regarding language model robustness in clinical applications. Finally\, we will showcase ongoing efforts to proactively address systemi c inequality in healthcare by identifying and characterizing stigmatizing language in medical records. DTSTART;TZID=America/New_York:20240226T120000 DTEND;TZID=America/New_York:20240226T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Keith Harrigian (JHU) “Fighting Bias From Bias: Robust Natural Lang uage Processing Techniques to Promote Health Equity” URL:https://www.clsp.jhu.edu/events/keith-harrigian-jhu-fighting-bias-from- bias-robust-natural-language-processing-techniques-to-promote-health-equit y/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\n
Abstr act
\nAs artificial intelligence (AI) continues to rapidly expand into existing healthcare infrastructure – e.g.\, clinical decision support\, administrative tasks\, and public health surveillance – it is pe rhaps more important than ever to reflect on the broader purpose of such s ystems. While much focus has been on the potential for this technology to improve general health outcomes\, there also exists a significant\, but un derstated\, opportunity to use this technology to address health-related d isparities. Accomplishing the latter depends not only on our ability to ef fectively identify addressable areas of systemic inequality and translate them into tasks that are machine learnable\, but also our ability to measu re\, interpret\, and counteract barriers in training data that may inhibit robustness to distribution shift upon deployment (i.e.\, new populations\ , temporal dynamics). In this talk\, we will discuss progress made along b oth of these dimensions. We will begin by providing background on the stat e of AI for promoting health equity. Then\, we will present results from a recent clinical phenotyping project and discuss their implication on prev ailing views regarding language model robustness in clinical applications. Finally\, we will showcase ongoing efforts to proactively address systemi c inequality in healthcare by identifying and characterizing stigmatizing language in medical records.
\n X-TAGS;LANGUAGE=en-US:2024\,February\,Harrigian END:VEVENT BEGIN:VEVENT UID:ai1ec-24491@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION: DTSTART;TZID=America/New_York:20240401T120000 DTEND;TZID=America/New_York:20240401T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Yuan Gong URL:https://www.clsp.jhu.edu/events/yuan-gong/ X-COST-TYPE:free X-TAGS;LANGUAGE=en-US:2024\,April\,Gong END:VEVENT BEGIN:VEVENT UID:ai1ec-24507@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION:Abstract\nHistory repeats itself\, sometimes in a bad way. Prev enting natural or man-made disasters requires being aware of these pattern s and taking pre-emptive action to address and reduce them\, or ideally\, eliminate them. Emerging events\, such as the COVID pandemic and the Ukrai ne Crisis\, require a time-sensitive comprehensive understanding of the si tuation to allow for appropriate decision-making and effective action resp onse. Automated generation of situation reports can significantly reduce t he time\, effort\, and cost for domain experts when preparing their offici al human-curated reports. However\, AI research toward this goal has been very limited\, and no successful trials have yet been conducted to automat e such report generation and “what-if” disaster forecasting. Pre-existing natural language processing and information retrieval techniques are insuf ficient to identify\, locate\, and summarize important information\, and l ack detailed\, structured\, and strategic awareness. In this talk I will p resent SmartBook\, a novel framework that cannot be solved by large langua ge models alone\, to consume large volumes of multimodal multilingual news data and produce a structured situation report with multiple hypotheses ( claims) summarized and grounded with rich links to factual evidence throug h multimodal knowledge extraction\, claim detection\, fact checking\, misi nformation detection and factual error correction. Furthermore\, SmartBook can also serve as a novel news event simulator\, or an intelligent prophe tess. Given “What-if” conditions and dimensions elicited from a domain ex pert user concerning a disaster scenario\, SmartBook will induce schemas f rom historical events\, and automatically generate a complex event graph a long with a timeline of news articles that describe new simulated events a nd character-centric stories based on a new Λ-shaped attention mask that c an generate text with infinite length. By effectively simulating disaster scenarios in both event graph and natural language format\, we expect Smar tBook will greatly assist humanitarian workers and policymakers to exercis e reality checks\, and thus better prevent and respond to future disasters .\nBio\nHeng Ji is a professor at Computer Science Department\, and an aff iliated faculty member at Electrical and Computer Engineering Department a nd Coordinated Science Laboratory of University of Illinois Urbana-Champai gn. She is an Amazon Scholar. She is the Founding Director of Amazon-Illin ois Center on AI for Interactive Conversational Experiences (AICE). She re ceived her B.A. and M. A. in Computational Linguistics from Tsinghua Unive rsity\, and her M.S. and Ph.D. in Computer Science from New York Universit y. Her research interests focus on Natural Language Processing\, especiall y on Multimedia Multilingual Information Extraction\, Knowledge-enhanced L arge Language Models\, Knowledge-driven Generation and Conversational AI. She was selected as a Young Scientist to attend the 6th World Laureates As sociation Forum\, and selected to participate in DARPA AI Forward in 2023. She was selected as “Young Scientist” and a member of the Global Future C ouncil on the Future of Computing by the World Economic Forum in 2016 and 2017. The awards she received include Women Leaders of Conversational AI ( Class of 2023) by Project Voice\, “AI’s 10 to Watch” Award by IEEE Intelli gent Systems in 2013\, NSF CAREER award in 2009\, PACLIC2012 Best paper ru nner-up\, “Best of ICDM2013” paper award\, “Best of SDM2013” paper award\, ACL2018 Best Demo paper nomination\, ACL2020 Best Demo Paper Award\, NAAC L2021 Best Demo Paper Award\, Google Research Award in 2009 and 2014\, IBM Watson Faculty Award in 2012 and 2014 and Bosch Research Award in 2014-20 18. She was invited to testify to the U.S. House Cybersecurity\, Data Anal ytics\, & IT Committee as an AI expert in 2023. She was invited by the Sec retary of the U.S. Air Force and AFRL to join Air Force Data Analytics Exp ert Panel to inform the Air Force Strategy 2030\, and invited to speak at the Federal Information Integrity R&D Interagency Working Group (IIRD IWG) briefing in 2023. She is the lead of many multi-institution projects and tasks\, including the U.S. ARL projects on information fusion and knowledg e networks construction\, DARPA ECOLE MIRACLE team\, DARPA KAIROS RESIN te am and DARPA DEFT Tinker Bell team. She has coordinated the NIST TAC Knowl edge Base Population task 2010-2022. She was the associate editor for IEEE /ACM Transaction on Audio\, Speech\, and Language Processing\, and served as the Program Committee Co-Chair of many conferences including NAACL-HLT2 018 and AACL-IJCNLP2022. She is elected as the North American Chapter of t he Association for Computational Linguistics (NAACL) secretary 2020-2023. Her research has been widely supported by the U.S. government agencies (DA RPA\, NSF\, DoE\, ARL\, IARPA\, AFRL\, DHS) and industry (Apple\, Amazon\, Google\, Facebook\, Bosch\, IBM\, Disney). DTSTART;TZID=America/New_York:20240405T120000 DTEND;TZID=America/New_York:20240405T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, Maryland 21218 SEQUENCE:0 SUMMARY:Heng Ji (University of Illinois Urbana-Champaign) “SmartBook: an AI Prophetess for Disaster Reporting and Forecasting” URL:https://www.clsp.jhu.edu/events/heng-ji-university-of-illinois-urbana-c hampaign-smartbook-an-ai-prophetess-for-disaster-reporting-and-forecasting / X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nHistory repeats itself\, sometimes in a bad way. Prev enting natural or man-made disasters requires being aware of these pattern s and taking pre-emptive action to address and reduce them\, or ideally\, eliminate them. Emerging events\, such as the COVID pandemic and the Ukrai ne Crisis\, require a time-sensitive comprehensive understanding of the si tuation to allow for appropriate decision-making and effective action resp onse. Automated generation of situation reports can significantly reduce t he time\, effort\, and cost for domain experts when preparing their offici al human-curated reports. However\, AI research toward this goal has been very limited\, and no successful trials have yet been conducted to automat e such report generation and “what-if” disaster forecasting. Pre-existing natural language processing and information retrieval techniques are insuf ficient to identify\, locate\, and summarize important information\, and l ack detailed\, structured\, and strategic awareness. In this talk I will p resent SmartBook\, a novel framework that cannot be solved by large langua ge models alone\, to consume large volumes of multimodal multilingual news data and produce a structured situation report with multiple hypotheses ( claims) summarized and grounded with rich links to factual evidence throug h multimodal knowledge extraction\, claim detection\, fact checking\, misi nformation detection and factual error correction. Furthermore\, SmartBook can also serve as a novel news event simulator\, or an intelligent prophe tess. Given “What-if” conditions and dimensions elicited from a domain ex pert user concerning a disaster scenario\, SmartBook will induce schemas f rom historical events\, and automatically generate a complex event graph a long with a timeline of news articles that describe new simulated events a nd character-centric stories based on a new Λ-shaped attention mask that c an generate text with infinite length. By effectively simulating disaster scenarios in both event graph and natural language format\, we expect Smar tBook will greatly assist humanitarian workers and policymakers to exercis e reality checks\, and thus better prevent and respond to future disasters .
\nBio
\nHeng Ji is a professor at Computer Science Department\, and an affiliated faculty member at Electrical and Co mputer Engineering Department and Coordinated Science Laboratory of Univer sity of Illinois Urbana-Champaign. She is an Amazon Scholar. She is the Fo unding Director of Amazon-Illinois Center on AI for Interactive Conversati onal Experiences (AICE). She received her B.A. and M. A. in Computational Linguistics from Tsinghua University\, and her M.S. and Ph.D. in Computer Science from New York University. Her research interests focus on Natural Language Processing\, especially on Multimedia Multilingual Information Ex traction\, Knowledge-enhanced Large Language Models\, Knowledge-driven Gen eration and Conversational AI. She was selected as a Young Scientist to at tend the 6th World Laureates Association Forum\, and selected to participa te in DARPA AI Forward in 2023. She was selected as “Young Scientist” and a member of the Global Future Council on the Future of Computing by the Wo rld Economic Forum in 2016 and 2017. The awards she received include Women Leaders of Conversational AI (Class of 2023) by Project Voice\, “AI’s 10 to Watch” Award by IEEE Intelligent Systems in 2013\, NSF CAREER award in 2009\, PACLIC2012 Best paper runner-up\, “Best of ICDM2013” paper award\, “Best of SDM2013” paper award\, ACL2018 Best Demo paper nomination\, ACL20 20 Best Demo Paper Award\, NAACL2021 Best Demo Paper Award\, Google Resear ch Award in 2009 and 2014\, IBM Watson Faculty Award in 2012 and 2014 and Bosch Research Award in 2014-2018. She was invited to testify to the U.S. House Cybersecurity\, Data Analytics\, & IT Committee as an AI expert in 2 023. She was invited by the Secretary of the U.S. Air Force and AFRL to jo in Air Force Data Analytics Expert Panel to inform the Air Force Strategy 2030\, and invited to speak at the Federal Information Integrity R&D Inter agency Working Group (IIRD IWG) briefing in 2023. She is the lead of many multi-institution projects and tasks\, including the U.S. ARL projects on information fusion and knowledge networks construction\, DARPA ECOLE MIRAC LE team\, DARPA KAIROS RESIN team and DARPA DEFT Tinker Bell team. She has coordinated the NIST TAC Knowledge Base Population task 2010-2022. She wa s the associate editor for IEEE/ACM Transaction on Audio\, Speech\, and La nguage Processing\, and served as the Program Committee Co-Chair of many c onferences including NAACL-HLT2018 and AACL-IJCNLP2022. She is elected as the North American Chapter of the Association for Computational Linguistic s (NAACL) secretary 2020-2023. Her research has been widely supported by t he U.S. government agencies (DARPA\, NSF\, DoE\, ARL\, IARPA\, AFRL\, DHS) and industry (Apple\, Amazon\, Google\, Facebook\, Bosch\, IBM\, Disney).
\n X-TAGS;LANGUAGE=en-US:2024\,April\,Ji END:VEVENT BEGIN:VEVENT UID:ai1ec-24509@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION: DTSTART;TZID=America/New_York:20240408T120000 DTEND;TZID=America/New_York:20240408T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Berrak Sisman URL:https://www.clsp.jhu.edu/events/berrak-sisman/ X-COST-TYPE:free X-TAGS;LANGUAGE=en-US:2024\,April\,Sisman END:VEVENT BEGIN:VEVENT UID:ai1ec-24511@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION: DTSTART;TZID=America/New_York:20240412T120000 DTEND;TZID=America/New_York:20240412T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Sonal Joshi (JHU) URL:https://www.clsp.jhu.edu/events/sonal-joshi-jhu/ X-COST-TYPE:free X-TAGS;LANGUAGE=en-US:2024\,April\,Joshi END:VEVENT BEGIN:VEVENT UID:ai1ec-24515@www.clsp.jhu.edu DTSTAMP:20240329T061820Z CATEGORIES;LANGUAGE=en-US:Seminars CONTACT: DESCRIPTION: DTSTART;TZID=America/New_York:20240415T120000 DTEND;TZID=America/New_York:20240415T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Matthew Wipperman (Regeneron) URL:https://www.clsp.jhu.edu/events/matthew-wipperman-regeneron/ X-COST-TYPE:free X-TAGS;LANGUAGE=en-US:2024\,April\,Wipperman END:VEVENT END:VCALENDAR