Prototype-based semantic consistency learning for unsupervised 2D image-based 3D shape retrieval

In this paper, we study the task of unsupervised 2D image-based 3D shape retrieval (UIBSR), which aims to retrieve unlabeled shapes (target domain) using labeled images (source domain). Previous works on UIBSR mainly focus on aligning the prototypes generated by the source labels and predicted target pseudo labels for reducing the cross-domain discrepancy. However, simply maintaining consistency between features may corrupt the original semantic information. Moreover, the existing methods usually ignore the diversity of the instances during the adaptation process, which results in reducing the discrimination of features. To solve these problems, we propose the prototype-based semantic consistency (PSC) learning method, exploring semantic knowledge in both prototype-prototype and prototype-instance relationships in the probability space rather than the embedding space to preserve the structure of semantic information. Besides, we propose a novel adversarial scheme between feature extractor and classifier to explore the characteristic of different instances, which can further enhance the model to learn more robust representations. Extensive experiments on two challenging datasets demonstrate the superiority of our proposed method.