A deep learning based system for mathematical expression detection and recognition in document images

Detection and recognition of mathematical expressions in document images are two key steps for the development of a mathematical expression retrieval system. So far, many researches have proposed for the recognition of expressions. However, few systems have integrated the detection and recognition of expressions in document images. This paper presents a deep learning based system for mathematical expression detection and recognition. Firstly, mathematical expressions have been detected on document images using the You Only Look Once (YOLO) v3 network. Then, detected expressions have been recognized in an end-to-end way using the advanced neural network that is the Watch, Attend and Parse (WAP). The proposed system has been tested on the Marmot public dataset. The obtained accuracies of the detection of isolated and inline expressions are 93% and 73%, respectively. Meanwhile, accuracies of the recognition for isolated and detected expressions are 51.77% and 45.50%, respectively. The results have shown the promising application of our preliminary research.