Cross-modal Retrieval with Label Completion

Cross-modal retrieval has been attracting increasing attention because of the explosion of multi-modal data, e.g., texts and images. Most supervised cross-modal retrieval methods learn discriminant common subspaces minimizing the heterogeneity of different modalities by exploiting the label information. However, these methods neglect the fact that, in practice, the given labels of training data might be incomplete (i.e., some of their labels are missing). The low-quality labels result in less effective subspace and consequent unsatisfactory retrieval performance. To tackle this, we propose a novel model that simultaneously performs label completion and cross-modal retrieval. Specifically, we assume the to-be-learned common subspace can be jointly derived through two aspects: 1) linear projection from modality-specific features and 2) enriching mapping from the incomplete labels. We thus formulate the subspace learning problem as a co-regularized learning framework based on multi-modal features and incomplete labels. Extensive experiments on two large-scale multi-modal datasets demonstrate the superiority of our model for both label completion and cross-modal retrieval over the state-of-the-arts.