MM-UNet: A novel cross-attention mechanism between modules and scales for brain tumor segmentation

Brain tumor segmentation is an essential issue in medical image segmentation. However, it is still challenging to consider the relationship between modules and efficiently fuse the features between adjacent scales. In this paper, we propose a novel cross-attention network for brain tumor segmentation, namely multi-scales and multi-modules cross-attention UNet (MM-UNet), which contains two mechanisms, module and scale cross- attentions. The module cross-attention (MCA) MCA ) strategy connects and exchanges global information between adjacent modules. The scale cross-attention (SCA) SCA ) strategy has two policies, the scale-related non-local relationship module ( SCA SNR ) and the scale-related channel-based relationship module ( SCA SCR ), that fuses the information between adjacent scales to mix the multi-scale information. Experiments on well-known tumor datasets, BraTS 2020, which has 369 cases, and has been classified into training, validation, and testing sets with 17,576, 4395, and 5735 images, to evaluate the performance by segmenting three regions, the whole tumor area (WT), core tumor area (CT) and enhancing tumor area (ET). Moreover, we consider three numerical metrics, dice, precision, and Hausdorff metrics, and various visualization results to objectively evaluate and intuitively display the experimental results. The proposed model surpasses state-of-the-art methods and achieves 0.8519, 0.8889, and 1.2647 with a base version network in dice, precision, sensitivity, and Hausdorff metrics, respectively. Moreover, we demonstrate the visualization with segmentation results and heatmaps in various scenarios to present the robustness of the proposed network in each region.