Combining CNNs with Transformer for Multimodal 3D MRI Brain Tumor Segmentation

We apply an ensemble of modified TransBTS, nnU-Net, and a combination of both for the segmentation task of the BraTS 2021 challenge. We change the original architecture of the TransBTS model by adding Squeeze-and-Excitation blocks, increasing the number of CNN layers, replacing positional encoding in the Transformer block with a learnable Multilayer Perceptron (MLP) embeddings, which makes Transformer adjustable to any input size during inference. With these modifications, we can improve TransBTS performance largely. Inspired by a nnU-Net framework, we decided to combine it with our modified TransBTS by changing the architecture inside nnU-Net to our custom model. On the Validation set of BraTS 2021, the ensemble of these approaches achieves 0.8496, 0.8698, 0.9256 Dice score and 15.72, 11.057, 3.374 HD95 for enhancing tumor, tumor core, and whole tumor, correspondingly. On test set we get Dice score 0.8789, 0.8759, 0.9279, and HD95: 10.426, 17.203, 4.93. Our code is publicly available.