Grammatical Trigrams
John Lafferty
School of Computer Science
Carnegie Mellon University
November 14, 1995
========================================================
ABSTRACT
It is widely believed among speech and language researchers that the
incorporation of linguistic information should improve statistical
models of natural language and benefit applications such as speech
recognition. This belief has yet to be realized. In this talk I will
discuss some previous attempts for building more effective language
models and present some new ideas that this past work suggests. In
particular, I will give an overview of current work at CMU to develop
language modeling techniques that combine grammatical information with
n-gram statistics. This work uses link grammar, a form of dependency
grammar, to extract structural information, and both decision trees and
maximum entropy methods to estimate probabilities. After introducing
the relevant concepts, I will discuss areas of recent work that make
this approach more feasible. In particular, I will present techniques
that help decrease the computational burden of maximum entropy
estimation, and robust parsing algorithms that enable the approach
to be applied to disfluent and ungrammatical speech.
**************************************************************************