Multi-label classification of line chart images using convolutional neural networks

In this paper, we propose a new convolutional neural network (CNN) architecture to build a multi-label classifier that categorizes line chart images according to their characteristics. The class labels are organized in the form of trend property (increasing or decreasing) and functional property (linear or exponential). In the proposed method, the Canny edge detection technique is applied as a data preprocessing step to increase both the classification accuracy and training speed. In addition, two different multi-label solution approaches are compared: label powerset (LP) and binary relevance (BR) methods. The experimental studies show that the proposed LP-CNN model achieves 93.75% accuracy, while the BR-CNN model reaches 92.97% accuracy on the test set, which contains real-world line chart images. The aim of this study is to build an efficient classifier that can be used for many purposes, such as automatically captioning the chart images, providing recommendations, redesigning charts, organizing a collection of chart images and developing better search engines.