Deep ranking structural support vector machine for image tagging

Image tagging is an active research topic in computer vision and machine learning, due to its wide applications on semantic search and image retrieval. Although recent approaches based on deep neural networks can learn better representations to significantly boost performance, no structural information has been exploited. For example, "sky" and "ground" both have higher probability to appear simultaneously in the same outdoor scene. In this paper, we propose a deep ranking structural Support Vector Machine (RSSVM) for image tagging. We exploit deep learning for representation learning and propose a new ranking function over the learned features with label correlation. Specifically, we incorporate the global context information between labels into our ranking function and then formulate the multi-labeling problem as a ranking problem to handle structured output prediction. We transfer parameters from the existed convolutional neural network (CNN) model, and add additional two fully connected layers to build our deep neural structure. We evaluate our method on three widely used datasets, and show promising results over competitive baselines. (c) 2017 Elsevier B.V. All rights reserved.