Unequal adaptive visual recognition by learning from multi-modal data

Conventional domain adaptation tries to leverage knowledge obtained from the single source domain to recognize the data in the target domain, where only one modality exists in the source domain. This neglects the scenario that source domain can be acquired from multi-modal data, such as RGB data and depth data. In addition, conventional domain adaptation approaches generally assume source and target domains have the identical number of categories, which is quite restrict for real-world applications. In practice, the number of categories in the target domain is often less than that in the source domain. In this work, we focus on a more practical and challenging task that recognizes RGB data by learning from RGB-D data under an unequal label scenario, which suffers from three challenges: i) the addition of depth information, ii) the domain mismatch problem and iii) the negative transfer caused by unequal label numbers. Our main contribution is a novel method, referred to as unequal Distribution Visual-Depth Adaption (uDVDA), which takes advantage of depth data and handles domain mismatch problem under label inequality, simultaneously. Experiments show that uDVDA outperforms state-of-the-art models on different datasets, especially under unequal label scenario.(c) 2022 Elsevier Inc. All rights reserved.