Learning and Adapting Diverse Representations for Cross-domain Few-shot Learning

Conventional few-shot learning (FSL) mainly focuses on transferring knowledge from a single source dataset to a recognition scenario with only a few training samples but still similar to the source domain. In this paper, we consider a more practical FSL setting where multiple semantically different datasets are available to address a wide range of FSL tasks, especially for some recognition scenarios beyond natural images, such as aerial and medical imagery. It can be referred to as multi-source cross-domain FSL. To tackle the problem, we propose a two-stage learning scheme, termed learning and adapting multi-source representations (LAMR). In the first stage, we propose a multi-head network to obtain efficient multidomain representations, where all source domains share the same backbone except for the last parallel projection layers used for domain specialization. We train the representations in a multitask setting where each in-domain classification task is realized by a cosine classifier. In the second stage, considering that instance discrimination and class discrimination are crucial for robust recognition, we propose two contrastive objectives for adapting the pre-trained representations to be task-specialized on the fewshot data. Careful ablation studies verify that LAMR significantly improves representation transferability, showing consistent performance boosts for FSL. Experiments on the BSCD-FSL benchmarks demonstrate that LAMR can achieve state-of-theart results, highlighting its versatility and effectiveness for FSL of both natural and specific imaging.