Constrained Estimation of Hidden Markov Models
Dr. Mark Gales of IBM's T.J. Watson Research Center
at the CLSP/JHU Summer Research Workshop on August 12, 1998 at
10:30 am, Arellano Theater, Levering Hall.
Constrained Estimation of Hidden Markov Models
The use of constraints for estimating hidden Markov models has been
widespread in the field of speech recognition. For example, the most commonly
used constraint is parameter tying. Here, many models are assumed to have
the same parameters. The data from all these models are combined to estimate
a single "tied" model. These constraints serve two purposes. First they
allow the robust estimaton of model parameters by reducing the number of
free parameters. In addition they allow parameters for "unseen" models
to be estimated. This is simply achieved by, for example, forcing all unseen
models to be tied to observed models. Simple maximum likelihood re-estimation
formulae for these tied estimation cases have been derived and succesfully
applied on very large vocabulary tasks.
In recent years more flexible forms of constraints have been proposed.
Simple tying is a very crude constraint, particularly in
situations where there may be very little data to train the models,
for example speaker and environment adaptation. Here there may be a large
number of "unseen" models, so simply tying these to seen models degrades
performance. This talk will describe more recent forms of constraints that
have been applied in speech recognition. These constraints, typically linear,
allow large numbers of model parameters to be estimates using very little
data, whilst removing the constraint
that large numbers of the estimated parameters are tied. In particular
the application of linear contraints for covariance matrix modelling, speaker
adaptation and in speaker adaptive training will be discussed.
Click Here for a Preliminary Look at his Presentation