Dual Channel-Spatial Self-Attention Transformer and CNN synergy network for 3D medical image segmentation

Even though the Vision Transformer leverages the self-attention mechanism to capture long-range dependencies, showing significant potential in medical image segmentation, the limited annotations in the image dataset make it difficult for the Transformer model to extract different global features, resulting in attention collapse and generating similar or identical attention maps. Previous studies have attempted to solve the problem by integrating convolutional neural layers into Transformer-based architectures. However, improper integration may lead to the inability of the model to effectively capture local and global information in both spatial and channel dimensions. To address the above issue, we propose a hybrid architecture using the Dual Channel-Spatial SelfAttention Transformer and CNN Synergy Network (DTC-SUNETR) for medical image segmentation. Specifically, we redesigned the self-attention mechanism. A novel Channel-Spatial Self-Attention (CSSA) block is introduced to integrate the enhanced channel and spatial self-attention mechanism to capture the global relationship and local structure among image features. This helps the model to more comprehensively understand the interdependencies between different channels and capture the relationships between different pixels, thus enhancing the feature representation of the corresponding dimensions. Simultaneously, it also improves the overall computational efficiency of the network. Extensive experiments on four different medical image segmentation datasets, including Synapse, ACDC, Brain Tumor, and Lung Tumor, demonstrate the superiority of the proposed DTC-SUNETR over state-of-the-art methods.