The internationally-recognized summer workshop brings researchers and students together to discuss challenges in multimodal AI, conversational systems, and health technology

More than 50 researchers and students from around the world will gather at Johns Hopkins University from June 15 to August 7 for eight weeks of collaborative research in speech, language, and artificial intelligence. 

Participants in the 12th annual Frederick Jelinek Memorial Summer Workshop on Speech and Language Technology, which is hosted by the Center for Language and Speech Processing (CLSP), gathers experts in the field to solve a set of challenges.

Previous workshops have contributed to improvements in multilingual speech recognition, tools for documenting and preserving endangered languages, and multimodal systems that combine speech with images and other forms of data. Workshop projects have also produced widely used research resources, software, and publications that continue to shape the field.

Founded at Rutgers University in 1993 and hosted by Johns Hopkins experts since 1995, the workshop has become a leading forum of the international speech and language technology community. In 2014, it was renamed in honor of Frederick Jelinek, CLSP’s founding director and a pioneer in speech recognition research. Since then, the workshop has alternated between Johns Hopkins and partner institutions around the world, including the University of Washington, Carnegie Mellon University, École de Technologie Supérieure in Montreal and Le Mans Université in France.

Unlike traditional conferences, the summer workshop emphasizes long-term collaboration. Participants spend eight weeks working side-by-side on research projects, sharing expertise and exploring new directions in speech and language technology.

The workshops begin with a two-week summer school featuring lectures, seminars, and hands-on learning opportunities covering topics like large language models, natural language processing, multimodal learning and conversational AI. Participants then transition into six weeks of intensive team-based research focused on three projects selected through the workshop’s interactive peer-review process.

Three projects for this summer’s workshop include:

Omnimodal Encoders: A Neural Bridge to Combine Many Forms of Data Sensory Data 

This team will explore new approaches to building AI systems capable of understanding and integrating information across multiple forms of data, including speech, text, images, and video. By developing more versatile multimodal representations, researchers hope to advance AI systems that can better understand complex real-world information and interactions.

Simulator for Evaluating Spoken Conversational AI

As conversational AI tools, such as chatbots and voice assistants, become increasingly common, researchers need better ways to assess how these systems perform. This project will focus on creating simulation environments that can evaluate spoken conversational AI systems under a variety of conditions, helping researchers improve their reliability, safety, and effectiveness.

AI Enabled Health Coaching for Sustainable Habit Building with Human Oversight

This group will investigate how AI can support long-term health and wellness goals while recognizing the limits of automated decision-making. By incorporating uncertainty estimates—quantifying the limitations in predicting variable health outcomes—and human oversight into AI-assisted coaching systems, researchers aim to create tools that can help users build sustainable habits while maintaining appropriate levels of human involvement.

The workshop concludes in August with presentations from each research team summarizing their work and outlining potential directions for future research.