Combining Neural Networks and Context-Driven Search for On-Line, Printed Handwriting Recognition in the Newton

Larry Yaeger, Apple Computer, Inc.

February 17, 1998


While on-line handwriting recognition is an area of long-standing and ongoing research, the recent emergence of portable, pen-based computers has focused urgent attention on usable, practical solutions. I will discuss a combination and improvement of classical methods to produce robust recognition of hand-printed English text, for a recognizer shipping in new models of Apple Computer's Newton MessagePad and eMate. Despite the Newton's ignominious past, this second-generation "Print Recognizer" is widely regarded to have provided the world's first truly useable handwriting recognition system. A straightforward combination of an artificial neural network (ANN), as a character classifier, with a context-driven search over segmentation and word recognition hypotheses provides the basis of this recognition system. Long-standing issues relative to training, generalization, segmentation, models of context, probabilistic formalisms, etc., needed to be resolved, however, to obtain excellent performance. I will give an overview of the entire recognition architecture, plus present a number of recent innovations in the application of ANNs as character classifiers for word recognition, including integrated multiple representations, normalized output error, negative training, stroke warping, frequency balancing, error emphasis, and quantized weights. User-adaptation and cursive recognition extensions to this technology will be discussed briefly.


Larry Yaeger is a programmer-scientist who has made technical contributions in the fields of neural networks and handwriting recognition (Newton's "Print Recognizer"), artificial life (PolyWorld), user interface design (for Koko the gorilla), computer graphics (The Last Starfighter, 2010, and Labyrinth), and computational fluid dynamics. His efforts have aided the Newton Systems Group, the Apple Research Labs, and Alan Kay's Vivarium Program at Apple Computer, as well as Digital Productions, and various aerospace companies.