InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions

被引:288
作者
Wang, Wenhai [1 ]
Dai, Jifeng [1 ,2 ]
Chen, Zhe [1 ,3 ]
Huang, Zhenhang [1 ]
Li, Zhiqi [1 ,3 ]
Zhu, Xizhou [4 ]
Hu, Xiaowei [1 ]
Lu, Tong [3 ]
Lu, Lewei [4 ]
Li, Hongsheng [5 ]
Wang, Xiaogang [4 ,5 ]
Qiao, Yu [1 ]
机构
[1] Shanghai AI Lab, Shanghai, Peoples R China
[2] Tsinghua Univ, Beijing, Peoples R China
[3] Nanjing Univ, Nanjing, Peoples R China
[4] SenseTime Res, Hong Kong, Peoples R China
[5] Chinese Univ Hong Kong, Hong Kong, Peoples R China
来源
2023 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR) | 2023年
基金
中国国家自然科学基金;
关键词
D O I
10.1109/CVPR52729.2023.01385
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Compared to the great progress of large-scale vision transformers (ViTs) in recent years, large-scale models based on convolutional neural networks (CNNs) are still in an early state. This work presents a new large-scale CNN-based foundation model, termed InternImage, which can obtain the gain from increasing parameters and training data like ViTs. Different from the recent CNNs that focus on large dense kernels, InternImage takes deformable convolution as the core operator, so that our model not only has the large effective receptive field required for downstream tasks such as detection and segmentation, but also has the adaptive spatial aggregation conditioned by input and task information. As a result, the proposed InternImage reduces the strict inductive bias of traditional CNNs and makes it possible to learn stronger and more robust patterns with large-scale parameters from massive data like ViTs. The effectiveness of our model is proven on challenging benchmarks including ImageNet, COCO, and ADE20K. It is worth mentioning that InternImage-H achieved a new record 65.4 mAP on COCO test-dev and 62.9 mIoU on ADE20K, outperforming current leading CNNs and ViTs.
引用
收藏
页码:14408 / 14419
页数:12
相关论文
共 79 条
[1]  
[Anonymous], 2017, IEEE CVF C COMP VIS
[2]  
[Anonymous], 2021, ICCV, DOI DOI 10.1109/ICCV48922.2021.00305
[3]  
[Anonymous], 2022, ADV NEURAL INFORM PR, DOI DOI 10.1145/3529399.3529415
[4]  
[Anonymous], 2018, EUR C COMP VIS
[5]  
Ba JL., 2016, LAYER NORMALIZATION
[6]   Cascade R-CNN: High Quality Object Detection and Instance Segmentation [J].
Cai, Zhaowei ;
Vasconcelos, Nuno .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2021, 43 (05) :1483-1498
[7]   Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts [J].
Changpinyo, Soravit ;
Sharma, Piyush ;
Ding, Nan ;
Soricut, Radu .
2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, :3557-3567
[8]   DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs [J].
Chen, Liang-Chieh ;
Papandreou, George ;
Kokkinos, Iasonas ;
Murphy, Kevin ;
Yuille, Alan L. .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2018, 40 (04) :834-848
[9]  
Chen Qiang, 2022, arXiv preprint arXiv:220713085
[10]  
Chen Zhe, 2022, ARXIV220508534