MMYFnet: Multi-Modality YOLO Fusion Network for Object Detection in Remote Sensing Images

被引:0
|
作者
Guo, Huinan [1 ]
Sun, Congying [1 ,2 ]
Zhang, Jing [2 ]
Zhang, Wuxia [3 ]
Zhang, Nengshuang [1 ,2 ]
机构
[1] Chinese Acad Sci, Xian Inst Opt & Fine Mech, Xian 710119, Peoples R China
[2] Xian Univ Technol, Sch Automat & Informat Engn, Xian 710048, Peoples R China
[3] Xian Univ Posts & Telecommun, Sch Comp Sci & Technol, Xian 710121, Peoples R China
基金
中国国家自然科学基金;
关键词
cross-modality; cosine similarity; feature fusion; multi-spectral remote sensing imagery; dual-branch; object detection; SIMILARITY;
D O I
10.3390/rs16234451
中图分类号
X [环境科学、安全科学];
学科分类号
08 ; 0830 ;
摘要
Object detection in remote sensing images is crucial for airport management, hazard prevention, traffic monitoring, and more. The precise ability for object localization and identification enables remote sensing imagery to provide early warnings, mitigate risks, and offer strong support for decision-making processes. While traditional deep learning-based object detection techniques have achieved significant results in single-modal environments, their detection capabilities still encounter challenges when confronted with complex environments, such as adverse weather conditions or situations where objects are obscured. To overcome the limitations of existing fusion methods in terms of complexity and insufficient information utilization, we innovatively propose a Cosine Similarity-based Image Feature Fusion (CSIFF) module and integrate it into a dual-branch YOLOv8 network, constructing a lightweight and efficient target detection network called Multi-Modality YOLO Fusion Network (MMYFNet). This network utilizes cosine similarity to divide the original features into common features and specific features, which are then refined and fused through specific modules. Experimental and analytical results show that MMYFNet performs excellently on both the VEDAI and FLIR datasets, achieving mAP values of 80% and 76.8%, respectively. Further validation through parameter sensitivity experiments, ablation studies, and visual analyses confirms the effectiveness of the CSIFF module. MMYFNet achieves high detection accuracy with fewer parameters, and the CSIFF module, as a plug-and-play module, can be integrated into other CNN-based cross-modality network models, providing a new approach for object detection in remote sensing image fusion.
引用
收藏
页数:19
相关论文
共 50 条
  • [1] MA-YOLO: a multi-attention object detection network for remote sensing images
    Song, Qingzeng
    Hou, Maorui
    Xue, Yongjiang
    Yu, Jing
    JOURNAL OF ELECTRONIC IMAGING, 2024, 33 (01)
  • [2] MULTI-SCALE FEATURE FUSION NETWORK FOR OBJECT DETECTION IN VHR OPTICAL REMOTE SENSING IMAGES
    Zhang, Wenhua
    Jiao, Licheng
    Liu, Xu
    Liu, Jia
    2019 IEEE INTERNATIONAL GEOSCIENCE AND REMOTE SENSING SYMPOSIUM (IGARSS 2019), 2019, : 330 - 333
  • [3] Cross-modality attentive feature fusion for object detection in multispectral remote sensing imagery
    Fang Qingyun
    Wang Zhaokui
    PATTERN RECOGNITION, 2022, 130
  • [4] Transfer Learning for Object Detection in Remote Sensing Images with YOLO
    Devi, A.
    Reddy, K. Venkateswara
    Bangare, Sunil L.
    Pande, Deepti S.
    Balaji, S. R.
    Badhoutiya, Arti
    Shrivastava, Anurag
    JOURNAL OF ELECTRICAL SYSTEMS, 2024, 20 (03) : 980 - 989
  • [5] A Novel Multi-Model Decision Fusion Network for Object Detection in Remote Sensing Images
    Ma, Wenping
    Guo, Qiongqiong
    Wu, Yue
    Zhao, Wei
    Zhang, Xiangrong
    Jiao, Licheng
    REMOTE SENSING, 2019, 11 (07)
  • [6] SeMo-YOLO: A Multiscale Object Detection Network in Satellite Remote Sensing Images
    Li, Peng
    Che, Cheng
    2021 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2021,
  • [7] An Adaptive Attention Fusion Mechanism Convolutional Network for Object Detection in Remote Sensing Images
    Ye, Yuanxin
    Ren, Xiaoyue
    Zhu, Bai
    Tang, Tengfeng
    Tan, Xin
    Gui, Yang
    Yao, Qin
    REMOTE SENSING, 2022, 14 (03)
  • [8] A multi-scale small object detection algorithm SMA-YOLO for UAV remote sensing images
    Zhou, Shilong
    Zhou, Haijin
    Qian, Lei
    SCIENTIFIC REPORTS, 2025, 15 (01):
  • [9] BA-YOLO for Object Detection in Satellite Remote Sensing Images
    Wang, Kuilin
    Liu, Zhenze
    APPLIED SCIENCES-BASEL, 2023, 13 (24):
  • [10] Multi-Modality Image Fusion and Object Detection Based on Semantic Information
    Liu, Yong
    Zhou, Xin
    Zhong, Wei
    ENTROPY, 2023, 25 (05)