Abstract

Biases in datasets, or unintentionally introduced spurious cues, are a common source of misspecification in machine learning. Performant models trained on such data can gender stereotype or be brittle under distribution shift.  In this talk, we present several results in multimodal and question answering applications studying sources of dataset bias, and several mitigation methods.  We propose approaches where known dimensions of dataset bias are explicitly factored out of a model during learning, without needing to modify data. Finally, we ask whether dataset biases can be attributable to annotator behavior during annotation. Drawing inspiration from work in psychology on cognitive biases, we show certain behavioral patterns are highly indicative of the creation of problematic (but valid) data instances in question answering. We give evidence that many existing observations around how dataset bias propagates to models can be attributed to data samples created by annotators we identify.

Biography

Mark Yatskar is an Assistant Professor at University of Pennsylvania in the department of Computer and Information Science. He did his PhD at University of Washington co-advised by Luke Zettlemoyer and Ali Farhadi. He was a Young Investigator at the Allen Institute for Artificial Intelligence for several years working with their computer vision team, Prior. His work spans Natural Language Processing, Computer Vision, and Fairness in Machine Learning. He received a Best Paper Award at EMNLP for work on gender bias amplification, and his work has been featured in Wired and the New York Times.

Abstract

The growing power in computing and AI promises a near-term future of human-machine teamwork. In this talk, I will present my research group’s efforts in understanding the complex dynamics of human-machine interaction and designing intelligent machines aimed to assist and collaborate with people. I will focus on 1) tools for onboarding machine teammates and authoring machine assistance, 2) methods for detecting, and broadly managing, errors in collaboration, and 3) building blocks of knowledge needed to enable ad hoc human-machine teamwork. I will also highlight our recent work on designing assistive, collaborative machines to support older adults aging in place.

Biography

Chien-Ming Huang is the John C. Malone Assistant Professor in the Department of Computer Science at the Johns Hopkins University. His research focuses on designing interactive AI aimed to assist and collaborate with people. He publishes in top-tier venues in HRI, HCI, and robotics including Science Robotics, HRI, CHI, and CSCW. His research has received media coverage from MIT Technology Review, Tech Insider, and Science Nation. Huang completed his postdoctoral training at Yale University and received his Ph.D. in Computer Science at the University of Wisconsin–Madison. He is a recipient of the NSF CAREER award. https://www.cs.jhu.edu/~cmhuang/