TransSea: Hybrid CNN-Transformer With Semantic Awareness for 3-D Brain Tumor Segmentation

Accurate segmentation of brain tumors in multimodal magnetic resonance imaging (MRI) plays a crucial role in clinical quantitative assessments, diagnostic processes, and the planning of therapeutic strategies. Both convolutional neural networks (CNNs) with strong local information extraction capacities and Transformers with excellent global representation capacities have achieved remarkable performance in medical image segmentation. However, considering the inherent semantic disparities between local and global features, effectively combining convolutions and Transformers presents a significant challenge in medical image segmentation. To address this issue, through integrating the merits of these two paradigms in a well-designed encoder-decoder architecture, we propose a hybrid CNN-Transformer network with semantic awareness, named TransSea, for an accurate 3-D brain tumor segmentation task. Our network incorporates a semantic mutual attention (SMA) module at the encoding stage, seamlessly integrating global and local features. Furthermore, our design includes a multiscale semantic guidance (SG) module that introduces semantic priors in the encoder through semantic supervision, enabling focused segmentation in relevant areas. In the decoding process, a semantic integration (SI) module is presented to further integrate various feature mappings from the encoder and semantic priors, thereby enhancing the propagation of semantic information and achieving semantically aware querying. Extensive experiments on two brain tumor datasets, BraTS2020 and BraTS2021, demonstrate that our model significantly outperforms existing state-of-the-art methods. The source code of the proposed method will be made available at https://github.com/yuliu316316.