Multi-sequence brain tumor segmentation boosted by deep semantic features

Background The main task of deep learning (DL) based brain tumor segmentation is to get accurate projection from learned image features to their corresponding semantic labels (i.e., brain tumor sub-regions). To achieve this goal, segmentation networks are required to learn image features with high intra-class consistency. However, brain tumor are known to be heterogeneous, and it often causes high diversity in image gray values which further influences the learned image features. Therefore, projecting such diverse image features (i.e., low intra-class consistency) to the same semantic label is often difficult and inefficient. Purpose The purpose of this study is to address the issue of low intra-class consistency of image features learned from heterogeneous brain tumor regions and ease the projection of image features to their corresponding semantic labels. In this way, accurate segmentation of brain tumor can be achieved. Methods We propose a new DL-based method for brain tumor segmentation, where a semantic feature module (SFM) is introduced to consolidate image features with meaningful semantic information and enhance their intra-class consistency. Specifically, in the SFM, deep semantic vectors are derived and used as prototypes to re-encode image features learned in the segmentation network. Since the relatively consistent deep semantic vectors, diversity of the resulting image features can be reduced; moreover, semantic information in the resulting image features can also be enriched, both facilitating accurate projection to the final semantic labels. Results In the experiment, a public brain tumor dataset, BraTS2022 containing, multi-sequence MR images of 1251 patients is used to evaluate our method in the task of brain tumor sub-region segmentation, and the experimental results demonstrate that, benefiting from the SFM, our method outperforms the state-of-the-art methods with statistical significance (p<0.05$p<0.05$ using the Wilcoxon signed rank test). Further ablation study shows that the proposed SFM can yield an improvement in segmentation accuracy (Dice index) of up to 11% comparing with that without the SFM. Conclusions In DL-based segmentation, low intra-class consistency of learned image features degrades segmentation performance. The proposed SFM can effectively enhance the intra-class consistency with high-level semantic information, making the projection of image features to their corresponding semantic labels more accurate.