Detection Transformer with Multi-granularity Information

Recent advancements in object detection have harnessed the power of DETR-like detectors, leveraging self-attention mechanisms to effectively capture long-range dependencies, and have drawn concern from researchers. However, these methods often fail to detect fine-grained object details, leading to the misidentification or omission of small objects, and ultimately making the model achieve sub-optimal performance. To address this challenge, we introduce a novel plug-and-play lightweight component, that can be flexibly integrated into the encoder layer, to extract fine-grained information and model local relationships. Specifically, Our proposed component comprises two modules: Multi-granularity Information Extraction and Multi-granularity Information Fusion. The information extraction employs dilated convolutions and large convolution kernels to capture multi-granularity features through multiple parallel branches. Subsequently, the information fusion utilizes self-attention guidance to effectively fuse information extracted at different granularities. Extensive experiments on the COCO benchmark dataset demonstrate that our proposed method outperforms the state-of-the-art (SOTA) method in terms of both accuracy and efficiency. Moreover, our method exhibits improvements when incorporated into DETR-like models. We achieve a 48.2 AP on the COCO detection test-dev using ResNet-DC-R101. Code will be available soon.