2022 JSALT Plenary Talks

2022 Annual Frederick Jelinek Memorial Summer Workshop

Wednesday July 6th, 11 AM to 12 PM EDT

Title: A Short Introduction to Arabic Natural Language Processing (Presentation Recording)

Speaker: Nizar Habash, New York University, Abu Dhabi

Location: Hackerman B17 (preferred) and Zoom (available)
Abstract: Arabic poses many challenges for natural language processing. The Arabic language is not only morphologically rich, but also highly ambiguous. In addition to Standard Arabic, there are many dialects that vary widely. We discuss these different challenges and their interactions, and present a number of basic enabling language technologies and resources to address them.

Bio: Nizar Habash is a Professor of Computer Science at New York University Abu Dhabi (NYUAD), and the director of the Computational Approaches to Modeling Language (CAMeL) Lab. Before joining NYUAD in 2014, he was a research scientist at Columbia University’s Center for Computational Learning Systems. He received his PhD in Computer Science from the University of Maryland College Park in 2003. He has two bachelors degrees, one in Computer Engineering and one in Linguistics and Languages. He specializes in natural language processing and computational linguistics, and his research includes extensive work on machine translation, morphological analysis, and computational modeling of Arabic and its dialects. He has over 200 publications including a book entitled “Introduction to Arabic Natural Language Processing”.

Wednesday July 13th, 11 AM to 12 PM EDT

Title: Coupled Excitable Systems: How Cells Migrate (Presentation Recording)

Speaker: Pablo Iglesias, The Johns Hopkins University

Location: Hackerman B17 (preferred) and Zoom (available)

Abstract: The study of excitable systems, which arose from a desire to understand the behavior of neurons, has a rich theoretical history. When excitable systems exist in diffusible media, the activity of the system can propagate as a traveling wave. In the last decade, there have been numerous reports that many of the molecular regulators of cell migration cells display wave propagation consistent with the existence of an underlying excitable system. In this talk I will present a comprehensive model for cell migration that explains cell movement in terms of excitable behavior. The presence of an excitable system accounts for the spontaneous migration of unstimulated cells and the directed migration of cells guided by external cues. No prior knowledge of the biology will be assumed.

Bio: Pablo Iglesias is the Edward J. Shaefer Professor of Electrical and Computer Engineering at Johns Hopkins University. He got his PhD from Cambridge University in 1991 and his Bachelor’s degree from the University of Toronto in 1987. His research focuses on the use of control and information theory to study biological signal transduction pathways. Among his research interests are: understanding how cells interpret directional cues to guide cell motion, the regulatory mechanisms that control cell division, and the sensing and actuation that enable cells to maintain lipid homeostasis.

Wednesday July 20th, 10:30 AM to 11:15 PM EDT CANCELLED

Title: End-to-End multi-talker ASR from overlapping speech utterances using combined audio and visual features Speaker: Richard Rose, Google Research, New YorkLocation: Hackerman B17 (preferred) and Zoom (available)Abstract: Decoding text transcriptions from utterances with multiple overlapping speakers has been addressed by a number of researchers – most notably here at the 2020 JSALT Workshop! It is well known that human listeners use visual information to disambiguate utterances from multiple talkers in multi-party human-human interactions. This has motivated the work described in this presentation where audio is combined with visual representations for multi-talker speech recognition. Attention based mechanisms for fusing audio with aligned visual features in a multi-talker framework are described. The overall framework extends conventional recurrent neural network transducer (RNN-T) architectures to include a masking model for disambiguating overlapping speech along with multiple label encoders to encode transcripts from overlapping speakers. Performance comparisons will be presented between audio-visual and audio-only multi-talker models and single channel models on a simulated overlapping speech corpus. Hopefully, workshop participants can share ideas on how state-of-the-art pre-trained models could be leveraged to provide multi-talker capability as a down-stream task. Bio: Rick Rose has been a research scientist at Google in New York City since October, 2014. While at Google he has contributed to efforts in audio-visual speech processing, acoustic modeling for ASR, and far-field speech recognition. Before coming to Google, he served as a Professor of Electrical and Computer Engineering at McGill University in Montreal since 2004, as a member of research staff at AT&T Labs / Bell Labs, and member of staff at MIT Lincoln Labs. He received his PhD degree in Electrical Engineering from the Georgia Institute of Technology. He has been active in the IEEE Signal Processing Society and is an IEEE Fellow.

Wednesday July 20th, 11:00 AM to 12:00 PM EDT – NEW START TIME

Title: Speech Research at Scale

Speakers: Pavel Golik, Google Research, New York

Location: Hackerman B17 (preferred) and Zoom (available)Abstract: Pavel Golik is a research scientist working in Google Speech on modeling, infra and quality analysis. Together with his team he builds tools that orchestrate automatic training of ASR models in 80+ languages that serve a wide range of applications both on device and on servers. In this talk he will discuss some of the challenges of internationalization (i18n) and what methods enable Google to iterate quickly on experimental prototypes and to even the path to production.

Bio: Pavel Golik received his PhD in Computer Science from the RWTH Aachen University in Germany. Prior to joining Google he was a speech scientist at AppTek where he built customized cloud ASR services. Pavel’s research interests include multilingual modeling and quality analysis

Friday July 22th, 11:00 AM to 12:00 PM EDT

Title: Gaelic Language Technology: Ongoing Progress and Future Possibilities (Presentation Recording)

Speaker: William Lamb, University of Edinburgh, UK

Location: Hackerman B17 (preferred) and Zoom (available)Abstract: Only about 100 of the world’s roughly 7000 languages have provision in advanced language technologies, such as machine translation and speech-to-text systems (van Esch et al, LREC 2022). Given that it is an endangered language with less than 60k speakers, Scottish Gaelic has a surprising level of provision in this area. For instance, in terms of language technology research, Scottish Gaelic has 12th highest paper count per capita in the world for the period 2010–2020 (ibid: 5041). Over the past decade, researchers have developed: part-of-speech taggers, lemmatisers, machine translation systems, an orthographic normaliser, a text-to-speech system, a syntactic parser, a handwriting recogniser and, most recently, a speech-to-text system. This talk will outline the current state-of-the-art and how some of these tools are being used within education and applied research. It will then consider what is required before we can move towards next generation NLP and NLG systems.

Bio: William Lamb is a Senior Lecturer in Celtic and Scottish Studies, School of Literatures, Languages and Cultures, at the University of Edinburgh, concentrating on the fields of Scottish ethnology and Gaelic linguistics. Prior to joining the University of Edinburgh in 2010, he was a lecturer at the University of the Highlands and Islands and an Honorary Research Fellow of Glasgow University. Since 2021, he has also been a research affiliate of Edinburgh Futures Institute. His research interests lie within the two broad areas of Gaelic linguistics and Scottish oral tradition.

Mon. July 25th, 1:00 PM to 2:00 PM EDT

Title: Translating Dialectal Arabic Speech: Lessons from IWSLT 2022

Speakers: Kevin Duh and the Motley Crew

Location: Hackerman B17 (preferred) and Zoom (available)Abstract: In some communities, two dialects of the same language are used by speakers under different settings. For example, in the Arabic-speaking world, Modern Standard Arabic (MSA) is used as spoken and written language for formal communications (e.g., news broadcasts, official speeches, religion), whereas informal communication is carried out in local dialects such as Egyptian, Moroccan, and Tunisian. This phenomenon, called diglossia, poses unique challenges to speech translation. Often only the “high” dialect for formal communication has sufficient training data for building good ASR and MT systems; the “low” dialect for informal communication may not even be commonly written.

The IWSLT202 Dialect Speech Task focused on this setting: the goal is to build a Tunisian-to-English speech translation system using small amounts of Tunisian-English data and additional ASR and MT resources in Modern Standard Arabic. We will discuss our collective experience as organizers and participants working on this problem. What are the lessons learned? What are the promising future research directions? Is there any potential synergy with efforts addressed by teams at JSALT 2022? Our goal is to have an informal interative group discussion.

Bio: Kevin Duh is a Senior Research Scientist at the Johns Hopkins University Human Language Technology Center of Excellence, an Assistant Research Professor in the Department of Computer Science, and a member of the Center for Language and Speech Processing. Previously, he was Assistant Professor at the Nara Institute of Science and Technology (2012-2015) and Research Associate at NTT CS Labs (2009-2012). Kevin received his B.S. in 2003 from Rice University, and PhD in 2009 from the University of Washington, both in Electrical Engineering. His research interests lie at the intersection of Natural Language Processing and Machine Learning, in particular on areas relating to machine translation, semantics, and deep learning.

Kevin will orchestrate a tag-team presentation with Matthew Wiesner, Yannick Esteve and Brian Yan.

Wed. July 27th, 11:00 AM to 12:00 PM EDT

Title: Speech-to-Speech Translation Technology

Speaker: Ann Lee, Meta AI, New York

Location: Hackerman B17 (preferred) and Zoom (available)Abstract: Direct speech-to-speech translation (S2ST) aims at translating speech from one language into speech in another language without relying on text generation as an intermediate step. Compared to conventional cascaded approaches, which take advantage of automatic speech recognition (ASR), machine translation (MT) or end-to-end speech-to-text translation (S2T) followed by text-to-speech synthesis (TTS), direct S2ST has the advantage of faster inference and can support translation between languages without text writing systems. In this talk, I will cover our recent series of work on (1) advancing direct S2ST with self-supervised discrete representations for speech, (2) democratizing S2ST technology for supporting unwritten languages, and (3) tackling data scarcity issue with pre-training.

Bio: Ann Lee is a tech lead manager at Meta AI, working with a team focusing on speech translation. Her current research focuses are on speech representation learning, speech translation and speech synthesis. Before joining Meta, she received her Ph.D. degree from MIT spoken language systems group in 2016. Her thesis focused on mispronunciation detection in non-native speech for computer-assisted pronunciation training.

Fri. July 29th, 11:00 AM to 12:00 PM EDT

Title: Disinformation, Fact-Checking, Propaganda, Media Bias and Framing

Speakers: Firoj Alam, Qatar Computing Research Institute

Location: Hackerman B17 (preferred) and Zoom (available)Abstract: Internet and social media offer technological tools for anyone to become a content creator and to share information online. On the positive side, this has given rise to citizen journalism, thus enabling much faster dissemination of information compared to what was possible before with newspapers, radio, and TV. However, taking away the gate-keeping role of traditional media has left the public unprotected against biased, deceptive, and dis-informative content. We explore the general fact-checking pipeline and important elements thereof such as check-worthiness estimation, spotting previously fact-checked claims, and detection of persuasion techniques. This talk offers an overview of the emerging and interconnected research areas of disinformation, fact-checking, propaganda, and media bias detection.

Bio: Firoj Alam is a Scientist at the Qatar Computing Research Institute, HBKU. He received his PhD from the University of Trento, Italy, and has been working for more than ten years in Artificial Intelligence, Deep/Machine learning, Natural Language Processing, Social media content, Image Processing, and Conversation Analysis. His current research includes disinformation detection, fact-checking, multimodal propaganda detection, and fighting the COVID-19 infodemic in multiple languages. He has made significant contributions to developing AI based tools and resources to support humanitarian organizations during disaster events and to support the UN-OCHA in streamlining their Education Insecurity effort. He is also well known in the Bangla Language computing community for his numerous contributions to advance Bangla language computing research.

Monday Aug 1st, 11:00 AM to 12:00 PM EDT

Title: Controllable Neural Language Generation Speaker: Nancy F. Chen, Agency for Science, Technology, and Research, SingaporeLocation: Hackerman B17 (preferred) and Zoom (available)Abstract: Natural language generation fuels numerous applications, including machine translation, summarization, and conversational agents. Advancements in end-to-end neural modeling have injected many new research endeavors into language generation. However, while neural models are extremely capable of generating fluent and grammatically correct text, many challenges still remain in ensuring factual correctness to minimize hallucination so model outputs can be more readily usable in deployment. In this talk, we will share some case studies in summarization and dialogue technology to illustrate how we can better harness the potential of neural language generation. Bio: Nancy Chen is a laboratory head, principal investigator and senior scientist at the Institute for Infocomm Research (I2R), Agency for Science, Technology and Research, Singapore, where she leads research on conversational AI and language intelligence with applications in education, healthcare, journalism, and defense. Speech evaluation technology developed by her team is deployed at the Ministry of Education in Singapore to support home-based learning, and their low-resource spoken language processing system was one of the top performers in the NIST Open Keyword Search Evaluations (2013-2016). She has received numerous awards, including Singapore 100 Women in Tech (2021), Young Scientist Award at MICCAI 2021, Best Paper Award at SIGDIAL 2021, the 2020 P&G Connect + Develop Open Innovation Award, the 2019 L’Oréal Singapore For Women in Science National Fellowship, Best Paper at APSIPA ASC (2016), MOE Outstanding Mentor Award (2012), the Microsoft-sponsored IEEE Spoken Language Processing Grant (2011), and the NIH (National Institute of Health) Ruth L. Kirschstein National Research Award (2004-2008). Technology from her team has also resulted in spin-off companies such as nomopai to help engage customers with confidence and empathy. She received her Ph.D. from MIT and Harvard in 2011, and worked at MIT Lincoln Laboratory before joining I2R.

Tue. Aug 2nd, 11:00 AM to 12:00 PM EDT

Title: Deciphering speech: a top-down approach to zero-resource speech recognition

Speaker: Peter Bell, University of Edinburgh, UK

Location: Hackerman B17 (preferred) and Zoom (available)Abstract: In this talk I will present a new method for training an automatic speech recognition system for a language for which absolutely no transcribed training data is available – a so-called “zero resource” scenario. Our approach is inspired by traditional methods of decipherment used in code-breaking, where strong prior knowledge of the language in question can be leveraged to crack a code with only limited quantities of cipher text. When applying this technique to speech, we use cross-lingual knowledge transfer as a means of making the problem tractable, and find that new languages can be “deciphered” with as little as 20 minutes of audio, and no phonetic knowledge of the language in question.

Bio: Peter Bell is a Reader in Speech Technology in the School of Informatics at the University of Edinburgh, and a member of the Centre for Speech Technology Research. His research interests include all aspects of automatic speech recognition development for low-resource languages. He was previously a co-investigator on the IARPA MATERIAL project and is currently principal investigator on the Unmute project, which aims to develop spoken language interfaces for traditionally marginalized language communities.

Jan

Fri