A Multi-Modal Hashing Learning Framework for Automatic Image Annotation

Automatic Image Annotation (AIA) plays an important role in large-scaled intelligent image management and retrieval. Based on the correlation between image low-level features and high-level semantic concepts, images can be efficiently retrieved from large-scaled image dataset. Recently, many researchers leverage machine learning techniques to annotate images automatically. However, these methods still have many challenges regarding efficiency and scalability in the massive image dataset. Moreover, manually labeling massive images is a costly and time-consuming task, which is unacceptable in practical applications. Therefore, only a few labeled images can be obtained as samples in the training dataset. However, the tags associated with labeled and unlabeled images found on social network websites may be helpful for improving the performance of AIA. In this work, we propose a Multi-Modal Semantic Hash Learning framework named MMSHL for AIA. MMSHL seamlessly integrates multi-graph learning, multimodal correlation learning and latent semantic hashing learning into a joint optimization framework. Based on MMSHL, we annotate images using a two-step semi-supervised learning approach. Since our AIA method makes use of associated tags of images, good results can be achieved. Extensive experiments are performed based on two real-world datasets MIR Flickr and NUS-WIDE. Experimental results show that our framework can improve the performance of AIA effectively.