Adaptive Discrepancy Masked Distillation for remote sensing object detection

被引:0
|
作者
Li, Cong [1 ]
Cheng, Gong [1 ]
Han, Junwei [1 ]
机构
[1] Northwestern Polytech Univ, Sch Automat, Xian 710129, Peoples R China
基金
中国国家自然科学基金;
关键词
Object detection; Knowledge distillation; Remote sensing images;
D O I
10.1016/j.isprsjprs.2025.02.006
中图分类号
P9 [自然地理学];
学科分类号
0705 ; 070501 ;
摘要
Knowledge distillation (KD) has become a promising technique for obtaining a performant student detector in remote sensing images by inheriting the knowledge from a heavy teacher detector. Unfortunately, not every pixel contributes (even detrimental) equally to the final KD performance. To dispel this problem, the existing methods usually derived a distillation mask to stress the valuable regions during KD. In this paper, we put forth Adaptive Discrepancy Masked Distillation (ADMD), a novel KD framework to explicitly localize the beneficial pixels. Our approach stems from the observation that the feature discrepancy between the teacher and student is the essential reason for their performance gap. With this regard, we make use of the feature discrepancy to determine which location causes the student to lag behind the teacher and then regulate the student to assign higher learning priority to them. Furthermore, we empirically observe that the discrepancy masked distillation leads to loss vanishing in later KD stages. To combat this issue, we introduce a simple yet practical weight- increasing module, in which the magnitude of KD loss is adaptively adjusted to ensure KD steadily contributes to student optimization. Comprehensive experiments on DIOR and DOTA across various dense detectors show that our ADMD consistently harvests remarkable performance gains, particularly under a prolonged distillation schedule, and exhibits superiority over state-of-the-art counterparts. Code and trained checkpoints will be made available at https://github.com/swift1988.
引用
收藏
页码:54 / 63
页数:10
相关论文
共 50 条
  • [31] Negative-Core Sample Knowledge Distillation for Oriented Object Detection in Remote Sensing Image
    Zhang, Wenhui
    Zhang, Yidan
    Huang, Feilong
    Qi, Xiyu
    Wang, Lei
    Liu, Xiaoxuan
    IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2024, 62
  • [32] Consistency- and dependence-guided knowledge distillation for object detection in remote sensing images
    Chen, Yixia
    Lin, Mingwei
    He, Zhu
    Polat, Kemal
    Alhudhaif, Adi
    Alenezi, Fayadh
    EXPERT SYSTEMS WITH APPLICATIONS, 2023, 229
  • [33] Two-Way Assistant: A Knowledge Distillation Object Detection Method for Remote Sensing Images
    Yang, Xi
    Zhang, Sheng
    Yang, Weichao
    IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2024, 62 : 1 - 10
  • [34] Adaptive Downsampling and Scale Enhanced Detection Head for Tiny Object Detection in Remote Sensing Image
    Zhang, Yunzuo
    Liu, Ting
    Zhen, Jiawen
    Kang, Yaoxing
    Cheng, Yu
    IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, 2025, 22
  • [35] Multiscale Feature Knowledge Distillation and Implicit Object Discovery for Few-Shot Object Detection in Remote Sensing Images
    Chen, Jie
    Guo, Ya
    Qin, Dengda
    Zhu, Jingru
    Gou, Zhenbo
    Sun, Geng
    IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2025, 63
  • [36] Object-Centric Masked Image Modeling-Based Self-Supervised Pretraining for Remote Sensing Object Detection
    Zhang, Tong
    Zhuang, Yin
    Chen, He
    Chen, Liang
    Wang, Guanqun
    Gao, Peng
    Dong, Hao
    IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, 2023, 16 : 5013 - 5025
  • [37] LT-DETR: lightweight UAV object detection and dual knowledge distillation for remote sensing scenarios
    Han, Zixian
    Jia, Dongli
    Zhang, Lei
    MEASUREMENT SCIENCE AND TECHNOLOGY, 2025, 36 (03)
  • [38] Efficient Object Detection in Optical Remote Sensing Imagery via Attention-Based Feature Distillation
    Shamsolmoali P.
    Chanussot J.
    Zhou H.
    Lu Y.
    IEEE Transactions on Geoscience and Remote Sensing, 2023, 61 : 1 - 12
  • [39] An Adaptive Attention Fusion Mechanism Convolutional Network for Object Detection in Remote Sensing Images
    Ye, Yuanxin
    Ren, Xiaoyue
    Zhu, Bai
    Tang, Tengfeng
    Tan, Xin
    Gui, Yang
    Yao, Qin
    REMOTE SENSING, 2022, 14 (03)
  • [40] Adaptive Anchor Networks for Multi-Scale Object Detection in Remote Sensing Images
    Zhang, Miaohui
    Chen, Yunzhong
    Liu, Xianxing
    Lv, Bingxue
    Wang, Jun
    IEEE ACCESS, 2020, 8 : 57552 - 57565