Joint and Individual Feature Fusion Hashing for Multi-modal Retrieval

Unsupervised multi-modal hashing has received considerable attention in large-scale multimedia retrieval areas since its low storage and high search speed. Existing unsupervised multi-modal hashing methods usually aim to mine the complementary information and the structural information for different modalities and preserve them in low-dimensional discrete space. The main limitations are two folds: (1) The shared semantic properties and the specific-modality information between multi-modal data are not explored simultaneously, which limits the improvement of retrieval accuracy. (2) Most multi-modal hashing methods with rough fusion manners cause greatly the information loss. In this paper, we present an unsupervised Joint and Individual Feature Fusion Hashing (JIFFH) that jointly performs the unified feature learning and individual feature learning. A two-layer fusion architecture with an adaptive weighting scheme is adopted to fuse effectively the common semantic properties and the specific-modality data information. The experimental results on three public multi-modal datasets show that our proposed method is better than state-of-the-art unsupervised multi-modal hashing methods. In conclusion, the proposed JIFFH method is very effective to learn discriminative hash codes and can boost retrieval performance.