Abstract
How can we create an algorithm that learns to associate images with sentences in natural language that describe the situations depicted in them? This talk will describe ongoing research towards this goal, with a focus on the natural language understanding aspects. Although we believe that this task may benefit from improved object recognition and deeper linguistic analysis, we show that models that rely on simple perceptual cues of color, texture and local feature descriptors on the image side, and on sequence-based features on the text side, can do surprisingly well. We also demonstrate how to leverage the availability of multiple captions for the same image.
Biography
Julia Hockenmaier is assistant professor of computer science at the University of Illinois at Urbana-Champaign. She came to Illinois after a postdoc at the University of Pennsylvania and a PhD at the University of Edinburgh. She holds an NSF CAREER award.