A pipeline strategy for handwritten numeral recognition that combines a two-stage template-based technique and a model-based technique is described. The template matcher combines multiple information sources. The second stage of the template matcher was trained on rejects from the first stage. The template matcher classifies 70-80% of the digits with reliability rates over 99%. It also generates class membership hypotheses for the remaining digits which constrain the model-based system. Recognition rates of 94.03-96.39% and error rates of 0.54%-1.05% are obtained on test data consisting of over 13,000 well-segmented digits from ZIP codes in the USPS mail.