Multiple Kernel Learning Based on Weak Learner for Automatic Image Annotation

Image annotation is a challenging problem, which has attracted intensive attention recently due to the semantic gap between images and corresponding tags. However, most existing works neglect the imbalance distribution of different classes and the internal correlations across modalities. To address these issues, we propose a multiple kernel learning method based on weak learner for image annotation, which can acquire the semantic correlations to predict tags of a given image. More specifically, we first employ the convolutional neural network to extract the semantic features of images, and take advantage of the over-sampling technique to generate new samples of minority classes which can solve the imbalance problem. Further, our proposed multiple kernel learning method is applied to obtain the internal correlations between images and tags. In order to further improve the prediction performance, we combine the boosting procedure with the multiple kernel learning to enhance the performance of classifier. We evaluate the proposed method on two benchmark datasets. The experimental results demonstrate that our method is superior to several state-of-the-art methods.