Deep semantic similarity adversarial hashing for cross-modal retrieval

Cross-modal retrieval has attracted considerate attention due to the rapid development of Internet and social media, and cross-modal hashing has been widely and successfully used in this domain. However, most existing hashing methods consider much little the semantic similarity levels between instances, whereas simply classified the semantic relationship as either similar or dissimilar. Besides, the issue of preservation of semantic similarity of original data between the extracted features is less explored from the existing methods. Due to the heterogeneity between different modalities, similarity of different modality features cannot be calculated directly. Therefore, in this paper, we propose a deep semantic similarity adversarial hashing (DSSAH) for cross-modal retrieval. We first calculate semantic similarity by using both label and feature information to provide a more accurate value for similarity between instances. And then an adversarial modality discriminator is introduced to establish a common feature space where similarity of each modality features can be calculated. Finally, two loss functions referring to inter-modal loss and intra-modal loss are designed to generate high quality hash codes. Experiments on three common datasets for cross-modal retrieval show that DSSAH outperforms state-of-the-art cross-modal hashing methods in cross-modal retrieval applications. (C) 2020 Elsevier B.V. All rights reserved.