Boosting the deep multidimensional long-short-term memory network for handwritten recognition systems

One of the main challenges in the handwriting recognition area lies in identifying complete lines of handwritten text. In this paper, we propose a handwriting recognition system based on a deep multidimensional long-short-term memory (MDLSTM) network within a hybrid hidden Markov model framework. The MDLSTM architecture was elaborated to enhance the recognition performance and decrease the recognition time. Accordingly, we present modifications regarding the layers order and the number of pooling layers compared to a standard MDLSTM model. Since the results reported in the literature for deeper MDLSTM architectures relies on optimizing the network width with a fixed depth, we investigate the trade-off between both these properties to obtain an optimal topology. The system was evaluated with English handwritten text lines from the IAM database and the experiments demonstrated that the proposed MDLSTM architecture was able to maintain a robust recognition performance (around 3.6% CER and 10.5% WER) and present significant speedups, approximately 48% and 32% faster than the state-of-the-art MDLSTM optical model, regarding the learning and classification times, respectively. The full system including a decoder with linguistic knowledge presents competitive results with the state-of-the-art.