Offline handwritten word recognition using a hybrid neural network and Hidden Markov model

This paper describes an approach to combine neural network (NN) and Hidden Markov models (HMM) for solving handwritten word recognition problem. The preprocessing involves generating a segmentation graph that describes all possible ways to segment a word into letters. To recognize a word, the NN computes the observation probabilities for each letter hypothesis in the segmentation graph. The HMMs then compute the likelihood for each word in the lexicon by summing the probabilities over all possible paths through the graph. One critical criterion for the NN-HMM hybrid system is that the NN character recognizer should be able to recognize non-characters or junks, apart from having the ability to distinguish between characters. In other words, the NN should give low probabilities for all character classes if junks are presented. We introduce the discriminant training to train the NN to recognize junk. We present a structural training scheme to improve the performance of the recognizer. An offline handwritten word recognizer is developed based on this approach and the recognition performance of the recognizer on three isolated word image databases, namely, IRONOFF, SRTP and AWS, are presented.