Deep learning multimodal bilinear fusion network based on vibrational spectroscopy for diagnosis of benign and malignant thyroid tumors

被引：0
作者：
Zhou, Xuguang [1 ]
Chen, Xiangnan [1 ]
Song, Haitao [7 ,8 ]
Lv, Xiaoyi [1 ,3 ]
Gu, Jin [9 ,10 ]
Chen, Chen [1 ,2 ,3 ,4 ]
Chen, Cheng [1 ,5 ,6 ]
机构：
[1] Xinjiang Univ, Coll Software, Urumqi 830046, Xinjiang, Peoples R China
[2] Xinjiang Univ, Coll Informat Sci & Engn, Urumqi 830046, Peoples R China
[3] Xinjiang Univ, Key Lab Signal Detect & Proc, Urumqi 830046, Peoples R China
[4] Xinjiang Cloud Comp Applicat Lab, Karamay 834099, Peoples R China
[5] Peoples Hosp Xinjiang Uyghur Autonomous Reg, Dept Cardiol, Urumqi, Xinjiang, Peoples R China
[6] Xinjiang Key Lab Cardiovasc Homeostasis & Regenera, Urumqi, Xinjiang, Peoples R China
[7] Xinjiang Med Univ, Affiliated Tumor Hosp, Dept Breast & Thyroid Surg, Urumqi 830017, Xinjiang, Peoples R China
[8] Clin Med Res Ctr Breast & Thyroid Tumor Xinjiang, Urumqi 830017, Xinjiang, Peoples R China
[9] Tsinghua Univ, Inst Precis Med, BNRIST Bioinformat Div, MOE Key Lab Bioinformat, Beijing 100084, Peoples R China
[10] Tsinghua Univ, Dept Automat, Beijing 100084, Peoples R China
来源：
MICROCHEMICAL JOURNAL | 2025年 / 209卷
关键词：
Raman spectroscopy; FTIR spectroscopy; Bilinear fusion; Multimodal fusion; Diagnosis of thyroid neoplasms; RAMAN-SPECTROSCOPY; CLASSIFICATION;
D O I：
10.1016/j.microc.2025.112870
中图分类号：
O65 [分析化学];
学科分类号：
070302 ; 081704 ;
摘要：
Background: Thyroid tumors, a common neoplasm type, are classified into benign and malignant forms. Benign tumors include follicular adenoma and papillary cystadenoma, while malignant types cover papillary carcinoma, follicular carcinoma, medullary carcinoma, and anaplastic carcinoma. Accurate differentiation between benign and malignant thyroid tumors is essential for effective treatment and prognosis. In the era of big data, the complexity and variety of biological sample data have increased significantly, highlighting the potential of Raman and infrared spectroscopy for biological analysis. Deep learning techniques show superior performance over traditional machine learning in data preprocessing, feature extraction, spectral analysis, and highdimensional data modeling. Results: This study comprehensively examines the current research landscape and potential future directions for benign and malignant thyroid tumor diagnosis. We introduce a multimodal bilinear fusion network (MBFN) that integrates Raman and infrared spectroscopy data using deep learning for improved diagnostic accuracy. Our proposed MBFN architecture is based on convolutional neural networks (CNN) and bidirectional extended shortterm memory networks (BiLSTM), specifically designed to extract intra-modality spectral features while utilizing a bilinear fusion mechanism to enable interactive feature representation between modalities. Evaluated on a dataset comprising 196 thyroid tumor samples, the MBFN approach achieved an accuracy of 97.95 % and an area under the curve (AUC) of 97.97 %, surpassing conventional diagnostic approaches. This superior performance underscores the efficacy of MBFN in enhancing diagnostic accuracy, setting a new benchmark for thyroid tumor classification. Significance: The findings of this research contribute novel methodologies and perspectives for the early diagnosis of thyroid tumors. A deep learning multimodal fusion approach is particularly valuable for managing nonlinear data and integrating complementary information across modalities, thereby enhancing feature representation. The MBFN approach demonstrates the potential of multimodal deep learning frameworks in improving diagnostic accuracy and advancing thyroid tumor research, paving the way for broader applications in the biomedical diagnostic field.
引用
收藏
页数：13
相关论文
共 54 条
  • [21] Chugh G., Kumar S., Singh N., Survey on machine learning and deep learning applications in breast cancer diagnosis, Cogn. Comput., 13, 6, pp. 1451-1470, (2021)
  • [22] Kline A., Wang H., Li Y., Dennis S., Hutch M., Xu Z., Luo Y., Multimodal machine learning in precision health: a sco** review, npj Digital Med., 5, 1, (2022)
  • [23] Zhang S., Qi X., Gao M., Dai C., Yin G., Ma D., He L., Estimation of wheat protein content and wet gluten content based on fusion of hyperspectral and RGB sensors using machine learning algorithms, Food Chem., 448, (2024)
  • [24] Cai G., Zhu Y., Wu Y., Jiang X., Ye J., Yang D., A multimodal transformer to fuse images and metadata for skin disease classification, Vis. Comput., 39, 7, pp. 2781-2793, (2023)
  • [25] Lazli L., Boukadoum M., Mohamed O.A., A survey on computer-aided diagnosis of brain disorders through MRI based on machine learning and data mining methodologies with an emphasis on Alzheimer disease diagnosis and the contribution of the multimodal fusion, Appl. Sci., 10, 5, (2020)
  • [26] de Brito E.C., Franca T., Canassa T., Weber S.S., Paniago A.M., Cena C., Paracoccidioidomycosis screening diagnosis by FTIR spectroscopy and multivariate analysis, Photodiagn. Photodyn. Ther., 39, (2022)
  • [27] Chen Y., Mao Q., Wang B., Duan P., Zhang B., Hong Z., Privacy-preserving multi-class support vector machine model on medical diagnosis, IEEE J. Biomed. Health Inform., 26, 7, pp. 3342-3353, (2022)
  • [28] Reges O., Krefman A.E., Hardy S.T., Yano Y., Muntner P., Lloyd-Jones D.M., Allen N.B., Decision tree-based classification for maintaining normal blood pressure throughout early adulthood and middle age: findings from the coronary artery risk development in young adults (CARDIA) study, Am. J. Hypertens., 34, 10, pp. 1037-1041, (2021)
  • [29] Huang M., Theory and Implementation of Linear Regression, 2020 International Conference on Computer Vision, Image and Deep Learning (CVIDL), pp. 210-217, (2020)
  • [30] Mahfouz M.A., Shoukry A., Ismail M.A., EKNN: Ensemble classifier incorporating connectivity and density into kNN with application to cancer diagnosis, Artif. Intell. Med., 111, (2021)
123456