Content-based image retrieval using handcraft feature fusion in semantic pyramid

The main challenge of image retrieval systems is to retrieve similar samples in a way that can be interpreted in a semantic relationship with the user's query image. In recent years, deep neural networks, due to their remarkable role in extracting the content and semantic features of the image, have been at the center of attention for image retrieval. Processing occurs in deep neural networks, at multiple levels, and with a pyramid approach. This characteristic allows the extraction of semantic and high-level features from the image. On the other hand, the image content features can be extracted with high interpretability using handcraft features. Therefore, in the proposed approach, by fusing features and adding extra information sources, handcraft features are semantically enhanced. In this approach, handcraft features including color and texture are extracted from the semantic pyramid of the deep neural network. The semantic pyramid is the result of the fusion of feature maps in different levels of deep neural networks. Additionally, in this approach, feature vector interpretability is also considered. The t-SNE technique has been used to interpret the discriminability of the feature vector between the classes of the database. Also, the silhouette criterion has been introduced to study the degree of intra-class compatibility and inter-class dataset samples discriminability with feature vector.