Boosting image classification through semantic attention filtering strategies

Saliency Maps, frequently used to highlight significant information, can be combined with other paradigms, such as Bag of Visual Words (BoVW), to improve image description when the saliency regions correspond closely with the objects of interest. In this paper, we present three attention filtering strategies based on their saliency map that improve image classification using the BoVW framework, Spatial Pyramid Matching (SPM) and Convolutional Neural Networks (CNN) features. Firstly, we demonstrate how the blurring factor used in the Hou's image signature algorithm determines what information remains and impacts to the obtained accuracy in image classification. Next, we propose AutoBlur, a simple but effective approach to automatically select this factor. Then, based on AutoBlur, we introduce two variants of our approach SARF (Semantic Attention Region Filtering), to semantically remove non-relevant regions through a Mean Shift segmentation. The first one is based on the intersection of the Hou's image attention areas with its Mean Shift segmentation, while the second one discards regions using a key point voting system that relies on the Euclidean distance. The experiments carried out showed that the methods of Semantic Attention Filtering that we are proposing could be successfully used with both BoVW, SPM and CNN's in most of the evaluated situations. In the five datasets assessed, all the three proposed methods outperform the baseline when using BoVWs in almost every case. For Spatial Pyramid Matching, the behaviour is similar, finding that the baseline is superior to our proposals in only one of the datasets used. In the case of CNN's, our filtering proposal outperforms the baseline in two datasets, being very similar to it in the other cases. (c) 2018 Elsevier B.V. All rights reserved.