HAMMF: Hierarchical attention-based multi-task and multi-modal fusion model for computer-aided diagnosis of Alzheimer's disease

被引：1
作者：
Liu X. [1 ]
Li W. [1 ]
Miao S. [1 ]
Liu F. [2 ,3 ,4 ]
Han K. [5 ]
Bezabih T.T. [1 ]
机构：
[1] School of Computer Engineering and Science, Shanghai University, Shanghai
[2] Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen
[3] University of Chinese Academy of Sciences, Beijing
[4] BGI-Shenzhen, Shenzhen
[5] Medical and Health Center, Liaocheng People's Hospital, LiaoCheng
来源：
Computers in Biology and Medicine | 2024年 / 176卷
关键词：
Alzheimer's disease; Attention mechanism; Deep learning; Multi-modal fusion; Multi-task learning; Transformer;
D O I：
10.1016/j.compbiomed.2024.108564
中图分类号：
学科分类号：
摘要：
Alzheimer's disease (AD) is a progressive neurodegenerative condition, and early intervention can help slow its progression. However, integrating multi-dimensional information and deep convolutional networks increases the model parameters, affecting diagnosis accuracy and efficiency and hindering clinical diagnostic model deployment. Multi-modal neuroimaging can offer more precise diagnostic results, while multi-task modeling of classification and regression tasks can enhance the performance and stability of AD diagnosis. This study proposes a Hierarchical Attention-based Multi-task Multi-modal Fusion model (HAMMF) that leverages multi-modal neuroimaging data to concurrently learn AD classification tasks, cognitive score regression, and age regression tasks using attention-based techniques. Firstly, we preprocess MRI and PET image data to obtain two modal data, each containing distinct information. Next, we incorporate a novel Contextual Hierarchical Attention Module (CHAM) to aggregate multi-modal features. This module employs channel and spatial attention to extract fine-grained pathological features from unimodal image data across various dimensions. Using these attention mechanisms, the Transformer can effectively capture correlated features of multi-modal inputs. Lastly, we adopt multi-task learning in our model to investigate the influence of different variables on diagnosis, with a primary classification task and a secondary regression task for optimal multi-task prediction performance. Our experiments utilized MRI and PET images from 720 subjects in the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset. The results show that our proposed model achieves an overall accuracy of 93.15% for AD/NC recognition, and the visualization results demonstrate its strong pathological feature recognition performance. © 2024 Elsevier Ltd
引用
收藏
相关论文
共 48 条
  • [31] Qin Z., Liu Z., Guo Q., Zhu P., 3D convolutional neural networks with hybrid attention mechanism for early diagnosis of Alzheimer's disease, Biomed. Signal Process. Control, 77, (2022)
  • [32] Xie H., Zeng X., Lei H., Du J., Wang J., Zhang G., Cao J., Wang T., Lei B., Cross-attention multi-branch network for fundus diseases classification using SLO images, Med. Image Anal., 71, (2021)
  • [33] Banerjee S., Chaudhuri S., Et al., pp. 2695-2704, (2020)
  • [34] Vaswani A., Shazeer N., Parmar N., Uszkoreit J., Jones L., Gomez A.N., Kaiser L., Polosukhin I., Attention is all you need, Adv. Neural Inf. Process. Syst., 30, (2017)
  • [35] Dosovitskiy A., Beyer L., Kolesnikov A., Weissenborn D., Zhai X., Unterthiner T., Dehghani M., Minderer M., Heigold G., Gelly S., Et al., An image is worth 16x16 words: Transformers for image recognition at scale, (2020)
  • [36] Touvron H., Cord M., Douze M., Massa F., Sablayrolles A., Jegou H., Training data-efficient image transformers & distillation through attention, International Conference on Machine Learning, pp. 10347-10357, (2021)
  • [37] Carion N., Massa F., Synnaeve G., Usunier N., Kirillov A., Zagoruyko S., End-to-end object detection with transformers, Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part I 16, pp. 213-229, (2020)
  • [38] Tzourio-Mazoyer N., Landeau B., Papathanassiou D., Crivello F., Etard O., Delcroix N., Mazoyer B., Joliot M., Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain, Neuroimage, 15, 1, pp. 273-289, (2002)
  • [39] Wen H., Liu Y., Rekik I., Wang S., Chen Z., Zhang J., Zhang Y., Peng Y., He H., Multi-modal multiple kernel learning for accurate identification of tourette syndrome children, Pattern Recognit., 63, pp. 601-611, (2017)
  • [40] Miao S., Xu Q., Li W., Yang C., Sheng B., Liu F., Bezabih T.T., Yu X., MMTFN: Multi-modal multi-scale transformer fusion network for Alzheimer's disease diagnosis, Int. J. Imaging Syst. Technol., (2023)
12345