We describe a learning algorithm for mixture models that directly optimizes their performance as classifiers. Our algorithm retains the main virtues of the Expectation-Maximization algorithm -- its guarantee of monotonic improvement, and its absence of tuning parameters -- with the added advantage of optimizing a discriminative objective function. The parameter updates and convergence proofs are based on simple intuitions. Experiments show that the algorithm significantly improves the discrimination of classifiers initially trained as generative models.

This is joint work with Dan Lee, Lucent - Bell Labs.

Lawrence Saul is a principal technical staff member in the speech and image processing center of AT&T Labs - Research. He received his Ph.D. in physics from M.I.T. In 2002, he will be joining the faculty of the Department of Computer and Information Science at the University of Pennsylvania.