A CLIP Guided Model for Few-Shot Object Detection

Detecting novel categories in few-shot object detection models is a challenging task due to the limited annotations during training, which consequently restricts the expressive power of the extracted detection features. In this paper, our focus is on effectively leveraging prior information to enhance feature representation capability, leading to the proposal of a few-shot object detection method based on the Contrastive Language-Image Pre-training (CLIP) model, named Few-CLIP. We introduced an image-text association module to integrate text features from the CLIP text encoder and image features from the Few-CLIP backbone, thereby incorporating semantic information. Additionally, an Adapter was employed within the image-text association module to fine-tune features from the image encoder of CLIP for Few-CLIP, thereby introducing more generalized image information. Evaluation of Few-CLIP on the PASCAL VOC and COCO datasets conformed its effectiveness at detecting novel categories with limited annotations and achieving performance comparable to other state-of-the-art few-shot object detection methods.