ResMT: A hybrid CNN-transformer framework for glioma grading with 3D MRI

被引:1
作者
Cui, Honghao [1 ]
Ruan, Zhuoying [2 ]
Xu, Zhijian [1 ]
Luo, Xiao [1 ]
Dai, Jian [1 ]
Geng, Daoying [1 ,2 ]
机构
[1] Fudan Univ, Acad Engn & Technol, Shanghai 200433, Peoples R China
[2] Fudan Univ, Huashan Hosp, Dept Radiol, Shanghai 200040, Peoples R China
关键词
Glioma grading; Deep learning; Hybrid architecture; Attention mechanisms; Transformer; Magnetic resonance imaging; CENTRAL-NERVOUS-SYSTEM; NETWORK;
D O I
10.1016/j.compeleceng.2024.109745
中图分类号
TP3 [计算技术、计算机技术];
学科分类号
0812 ;
摘要
Accurate grading of gliomas is crucial for treatment strategies and prognosis. While convolutional neural networks (CNNs) have proven effective in classifying medical images, they struggle with capturing long-range dependencies among pixels. Transformer-based networks can address this issue, but CNN-based methods often perform better when trained on small datasets. Additionally, tumor segmentation is essential for classification models, but training an additional segmentation model significantly increases workload. To address these challenges, we propose ResMT, which combines CNN and transformer architectures for glioma grading, extracting both local and global features efficiently. Specifically, we designed a spatial residual module (SRM) where a 3D CNN captures glioma's volumetric complexity, and Swin UNETR, a pre-trained segmentation model, enhances the network without extra training. Our model also includes a multi-plane channel and spatial attention module (MCSA) to refine the analysis by focusing on critical features across multiple planes (axial, coronal, and sagittal). Transformer blocks establish long-range relationships among planes and slices. We evaluated ResMT on the BraTs19 dataset, comparing it with baselines and state-of-the-art models. Results demonstrate that ResMT achieves the highest prediction performance with an AUC of 0.9953, highlighting hybrid CNN-transformer models' potential for 3D MRI classification.
引用
收藏
页数:17
相关论文
共 50 条
  • [21] Polarformer: Optic Disc and Cup Segmentation Using a Hybrid CNN-Transformer and Polar Transformation
    Feng, Yaowei
    Li, Zhendong
    Yang, Dong
    Hu, Hongkai
    Guo, Hui
    Liu, Hao
    APPLIED SCIENCES-BASEL, 2023, 13 (01):
  • [22] A 3D reconstruction method based on multi-views of contours segmented with CNN-transformer for long bones
    Yunfei Ge
    Qing Zhang
    Yidong Shen
    Yuantao Sun
    Chongyang Huang
    International Journal of Computer Assisted Radiology and Surgery, 2022, 17 : 1891 - 1902
  • [23] CloudSwinNet: A hybrid CNN-transformer framework for ground-based cloud images fine-grained segmentation
    Shi, Chaojun
    Su, Zibo
    Zhang, Ke
    Xie, Xiongbin
    Zhang, Xiaoyun
    ENERGY, 2024, 309
  • [24] Rethinking Image Deblurring via CNN-Transformer Multiscale Hybrid Architecture
    Zhao, Qian
    Yang, Hao
    Zhou, Dongming
    Cao, Jinde
    IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, 2023, 72
  • [25] CNN-Transformer hybrid network for concrete dam crack patrol inspection
    Li, Mingchao
    Yuan, Jingyue
    Ren, Qiubing
    Luo, Qiling
    Fu, Junen
    Li, Zhitang
    AUTOMATION IN CONSTRUCTION, 2024, 163
  • [26] TimeDistributed-CNN-LSTM: A Hybrid Approach Combining CNN and LSTM to Classify Brain Tumor on 3D MRI Scans Performing Ablation Study
    Montaha, Sidratul
    Azam, Sami
    Rafid, A. K. M. Rakibul Haque
    Hasan, Md. Zahid
    Karim, Asif
    Islam, Ashraful
    IEEE ACCESS, 2022, 10 : 60039 - 60059
  • [27] A lightweight 3D UNet model for glioma grading
    Yu, Xuan
    Wu, Yaping
    Bai, Yan
    Han, Hui
    Chen, Lijuan
    Gao, Haiyan
    Wei, Huanhuan
    Wang, Meiyun
    PHYSICS IN MEDICINE AND BIOLOGY, 2022, 67 (15)
  • [28] Enhancing Winter Wheat Yield Estimation With a CNN-Transformer Hybrid Framework Utilizing Multiple Remotely Sensed Parameters
    Du, Jiangli
    Zhang, Yue
    Wang, Pengxin
    Tansey, Kevin
    Liu, Junming
    Zhang, Shuyu
    IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2025, 63
  • [29] CNN-TransNet: A Hybrid CNN-Transformer Network With Differential Feature Enhancement for Cloud Detection
    Ma, Nan
    Sun, Lin
    He, Yawen
    Zhou, Chenghu
    Dong, Chuanxiang
    IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, 2023, 20
  • [30] Hybrid CNN-Transformer Architecture for Efficient Large-Scale Video Snapshot Compressive Imaging
    Cao, Miao
    Wang, Lishun
    Zhu, Mingyu
    Yuan, Xin
    INTERNATIONAL JOURNAL OF COMPUTER VISION, 2024, 132 (10) : 4521 - 4540