Abstract
Theories of sentence comprehension have traditionally been based on evidence from reading studies, and have emphasized cognitive limitations and invariant parsing strategies. There is increasing evidence, however, that human language comprehension is highly adapted both to long-term experience and to immediate linguistic and non-linguistic context. In this talk, I will present a collection of recent experiments focusing on adaptation to immediate context in situated language comprehension. Using the “visual world” paradigm, which monitors eye movements in scenes during spoken comprehension, we have demonstrated the ability of the human language processor to rapidly adapt to and exploit diverse linguistic cues, prosody, as well as information from the immediate scene. These findings have led to the proposal of the “coordinated interplay account” of scene and utterance processing Knoeferle and Crocker, Cognitive Science, to appear, which argues for the rapid use, and even priority, of scene information during spoken comprehension. In order to model the rapid use of both experience and immediate context, we have developed a family of connectionist models based on Elmans Simple Recurrent Networks. These networks have been modified to include inputs not only for the utterance, but also for the current visual context, and have been trained to model the findings of five experiments. In addition to the general ability of the networks to successfully learn the task, they also model on-line behavior: a. anticipation of role and filler, b. frequency biases derived from training, c. early influence of depicted events, d. delayed disambiguation without the scene, and e. relative priority of scene information Mayberry, Crocker & Knoeferle, IJCNLP, 2005. In conclusion, I argue that the mechanisms that underlie language comprehension are fundamentally adaptive in nature: optimized to recover the most likely interpretation based on relevant linguistic and non-linguistic information sources. I further suggest that theories and models of human language comprehension must be expanded to explain on-line interaction with other cognitive and perceptual processes.

Biography
Prof. Dr. Matthew W. Crocker began his study of Computer Science in Canada at the University of New Brunswick BSc, 1986 and the University of British Columbia MSc, 1988, where he specialised in computational linguistics. In 1992, he received a Doctorate in Philosophy from the Department of Artificial Intelligence, at the University of Edinburgh, Scotland now the School of Informatics. Dr. Crocker then continued in Edinburgh, where he was a lecturer in Artificial Intelligence and Cognitive Science 1992-94, and subsequently an ESRC Research Fellow 1994-1998. In 1999, Dr. Crocker took up a position as Research Scientist at the University of the Saarland, Germany, and in January 2000, he was appointed to the newly established Chair in Psycholinguistics, in the Department of Computational Linguistics at that University.