MAFNet: Multimodal Asymmetric Fusion Network for Radar Echo Extrapolation

被引:0
|
作者
Pei, Yanle [1 ,2 ]
Li, Qian [1 ,2 ]
Wu, Yayi [1 ,2 ]
Peng, Xuan [1 ,2 ]
Guo, Shiqing [1 ,2 ]
Ye, Chengzhi [2 ,3 ]
Wang, Tianying [2 ,3 ]
机构
[1] Natl Univ Def Technol, Coll Meteorol & Oceanog, Changsha 410005, Peoples R China
[2] China Meteorol Adm CMA, High Impact Weather Key Lab, Changsha 410005, Peoples R China
[3] Inst Meteorol Sci Hunan Prov, Changsha 410118, Peoples R China
基金
中国博士后科学基金; 中国国家自然科学基金;
关键词
radar echo extrapolation (REE); convective nowcasting; multimodal data; asymmetric fusion; GENERATIVE ADVERSARIAL NETWORK; PRECIPITATION; EFFICIENT; MODEL;
D O I
10.3390/rs16193597
中图分类号
X [环境科学、安全科学];
学科分类号
08 ; 0830 ;
摘要
Radar echo extrapolation (REE) is a crucial method for convective nowcasting, and current deep learning (DL)-based methods for REE have shown significant potential in severe weather forecasting tasks. Existing DL-based REE methods use extensive historical radar data to learn the evolution patterns of echoes, they tend to suffer from low accuracy. This is because data of radar modality face difficulty adequately representing the state of weather systems. Inspired by multimodal learning and traditional numerical weather prediction (NWP) methods, we propose a Multimodal Asymmetric Fusion Network (MAFNet) for REE, which uses data from radar modality to model echo evolution, and data from satellite and ground observation modalities to model the background field of weather systems, collectively guiding echo extrapolation. In the MAFNet, we first extract overall convective features through a global shared encoder (GSE), followed by two branches of local modality encoder (LME) and local correlation encoders (LCEs) that extract convective features from radar, satellite, and ground observation modalities. We employ an multimodal asymmetric fusion module (MAFM) to fuse multimodal features at different scales and feature levels, enhancing radar echo extrapolation performance. Additionally, to address the temporal resolution differences in multimodal data, we design a time alignment module based on dynamic time warping (DTW), which aligns multimodal feature sequences temporally. Experimental results demonstrate that compared to state-of-the-art (SOTA) models, the MAFNet achieves average improvements of 1.86% in CSI and 3.18% in HSS on the MeteoNet dataset, and average improvements of 4.84% in CSI and 2.38% in HSS on the RAIN-F dataset.
引用
收藏
页数:22
相关论文
共 50 条
  • [1] MPFNet: Multiproduct Fusion Network for Radar Echo Extrapolation
    Pei, Yanle
    Li, Qian
    Zhang, Liang
    Sun, Nengli
    Jing, Jinrui
    Ding, Yuhong
    Shen, Hong
    Wang, Tianying
    IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2024, 62
  • [2] STUNNER: Radar Echo Extrapolation Model Based on Spatiotemporal Fusion Neural Network
    Fang, Wei
    Pang, Lin
    Sheng, Victor S.
    Wang, Qiguang
    IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2023, 61
  • [3] NLED: Nonlocal Echo Dynamics Network for Radar Echo Extrapolation
    Chen, Taisong
    Li, Qian
    Peng, Xuan
    Lv, Qi
    Jing, Jinrui
    IEEE Transactions on Geoscience and Remote Sensing, 2022, 60
  • [4] NLED: Nonlocal Echo Dynamics Network for Radar Echo Extrapolation
    Chen, Taisong
    Li, Qian
    Peng, Xuan
    Lv, Qi
    Jing, Jinrui
    IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2022, 60
  • [5] MAFNet: Segmentation of Road Potholes With Multimodal Attention Fusion Network for Autonomous Vehicles
    Feng, Zhen
    Guo, Yanning
    Liang, Qing
    Bhutta, M. Usman Maqbool
    Wang, Hengli
    Liu, Ming
    Sun, Yuxiang
    IEEE Transactions on Instrumentation and Measurement, 2022, 71
  • [6] MAFNet: Segmentation of Road Potholes With Multimodal Attention Fusion Network for Autonomous Vehicles
    Feng, Zhen
    Guo, Yanning
    Liang, Qing
    Bhutta, M. Usman Maqbool
    Wang, Hengli
    Liu, Ming
    Sun, Yuxiang
    IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, 2022, 71
  • [7] Learnable Optical Flow Network for Radar Echo Extrapolation
    Zhang, Chengwei
    Zhou, Xudong
    Zhuge, Xiaoyong
    Xu, Meng
    IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, 2021, 14 : 1260 - 1266
  • [8] Radar Echo Extrapolation Model Based on Deep Spatio-Temporal Fusion Neural Network
    Fang W.
    Pang L.
    Yi W.-N.
    Tien Tzu Hsueh Pao/Acta Electronica Sinica, 2023, 51 (09): : 2526 - 2538
  • [9] A Generative Adversarial and Spatiotemporal Differential Fusion Method in Radar Echo Extrapolation
    Niu, Xianghua
    Zhang, Lixia
    Wang, Chunlin
    Shen, Kailing
    Tian, Wei
    Liao, Bin
    REMOTE SENSING, 2023, 15 (22)
  • [10] Improved Weather Radar Echo Extrapolation Through Wind Speed Data Fusion Using a New Spatiotemporal Neural Network Model
    Geng, Huan-tong
    Xie, Bo-yang
    Ge, Xiao-yan
    Min, Jin-zhong
    Zhuang, Xiao-ran
    JOURNAL OF TROPICAL METEOROLOGY, 2023, 29 (04) : 482 - 492