The Fine-Tuned Large Language Model for Extracting the Progressive Bone Metastasis from Unstructured Radiology Reports

被引:2
作者
Kanemaru, Noriko [1 ]
Yasaka, Koichiro [1 ]
Fujita, Nana [1 ]
Kanzawa, Jun [1 ]
Abe, Osamu [1 ]
机构
[1] Univ Tokyo, Grad Sch Med, Dept Radiol, 7-3-1 Hongo Bunkyo-Ku, Tokyo 1138655, Japan
来源
JOURNAL OF IMAGING INFORMATICS IN MEDICINE | 2025年 / 38卷 / 02期
关键词
Large language model; Bone metastasis; Deep learning; DISEASE;
D O I
10.1007/s10278-024-01242-3
中图分类号
R8 [特种医学]; R445 [影像诊断学];
学科分类号
1002 ; 100207 ; 1009 ;
摘要
Early detection of patients with impending bone metastasis is crucial for prognosis improvement. This study aimed to investigate the feasibility of a fine-tuned, locally run large language model (LLM) in extracting patients with bone metastasis in unstructured Japanese radiology report and to compare its performance with manual annotation. This retrospective study included patients with "metastasis" in radiological reports (April 2018-January 2019, August-May 2022, and April-December 2023 for training, validation, and test datasets of 9559, 1498, and 7399 patients, respectively). Radiologists reviewed the clinical indication and diagnosis sections of the radiological report (used as input data) and classified them into groups 0 (no bone metastasis), 1 (progressive bone metastasis), and 2 (stable or decreased bone metastasis). The data for group 0 was under-sampled in training and test datasets due to group imbalance. The best-performing model from the validation set was subsequently tested using the testing dataset. Two additional radiologists (readers 1 and 2) were involved in classifying radiological reports within the test dataset for testing purposes. The fine-tuned LLM, reader 1, and reader 2 demonstrated an accuracy of 0.979, 0.996, and 0.993, sensitivity for groups 0/1/2 of 0.988/0.947/0.943, 1.000/1.000/0.966, and 1.000/0.982/0.954, and time required for classification (s) of 105, 2312, and 3094 in under-sampled test dataset (n = 711), respectively. Fine-tuned LLM extracted patients with bone metastasis, demonstrating satisfactory performance that was comparable to or slightly lower than manual annotation by radiologists in a noticeably shorter time.
引用
收藏
页码:865 / 872
页数:8
相关论文
共 43 条
  • [31] Understanding Citizens' Response to Social Activities on Twitter in US Metropolises During the COVID-19 Recovery Phase Using a Fine-Tuned Large Language Model: Application of AI
    Saito, Ryuichi
    Tsugawa, Sho
    JOURNAL OF MEDICAL INTERNET RESEARCH, 2025, 27
  • [32] Automatic structuring of radiology reports of chest X-ray images using large language model
    Nishio, Mizuho
    Matsuo, Hidetoshi
    Matsunaga, Takaaki
    2024 IEEE 48TH ANNUAL COMPUTERS, SOFTWARE, AND APPLICATIONS CONFERENCE, COMPSAC 2024, 2024, : 1952 - 1954
  • [33] Estimation of Beat-by-Beat Blood Pressure and Heart Rate From ECG and PPG Using a Fine-Tuned Deep CNN Model
    Yen, Chih-Ta
    Chang, Sheng-Nan
    Liao, Cheng-Hong
    IEEE ACCESS, 2022, 10 : 85459 - 85469
  • [34] Large Language Model Approach for Zero-Shot Information Extraction and Clustering of Japanese Radiology Reports: Algorithm Development and Validation
    Yamagishi, Yosuke
    Nakamura, Yuta
    Hanaoka, Shouhei
    Abe, Osamu
    JMIR CANCER, 2025, 11
  • [35] Event-Based Clinical Finding Extraction from Radiology Reports with Pre-trained Language Model
    Lau, Wilson
    Lybarger, Kevin
    Gunn, Martin L.
    Yetisgen, Meliha
    JOURNAL OF DIGITAL IMAGING, 2023, 36 (01) : 91 - 104
  • [36] Event-Based Clinical Finding Extraction from Radiology Reports with Pre-trained Language Model
    Wilson Lau
    Kevin Lybarger
    Martin L. Gunn
    Meliha Yetisgen
    Journal of Digital Imaging, 2023, 36 : 91 - 104
  • [37] Large Language Model-Assisted Information Extraction from Clinical Reports for Survival Prediction of Bladder Cancer Patients
    Sun, Di
    Hadjiiski, Lubomir
    Gormley, John
    Chan, Heang-Ping
    Caoili, Elaine M.
    Cohan, Richard H.
    Alva, Ajjai
    Mihalcea, Rada
    Zhou, Chuan
    Gulani, Vikas
    COMPUTER-AIDED DIAGNOSIS, MEDICAL IMAGING 2024, 2024, 12927
  • [38] Extracting Key Information from Unlabeled Patents Based on Knowledge Self-Distillation of Large Language Model
    Jianfei, Zhao
    Ting, Chen
    Xiaomei, Wang
    Chong, Feng
    Data Analysis and Knowledge Discovery, 2024, 8 (8-9) : 133 - 143
  • [39] Structuring Semi-structured Data from Building Inspection Reports Using a Large Language Model
    Svennberg, Kaisa
    Ekman, Jan
    MULTIPHYSICS AND MULTISCALE BUILDING PHYSICS, IBPC 2024, VOL 3, 2025, 554 : 508 - 513
  • [40] BreastNet18: A High Accuracy Fine-Tuned VGG16 Model Evaluated Using Ablation Study for Diagnosing Breast Cancer from Enhanced Mammography Images
    Montaha, Sidratul
    Azam, Sami
    Rafid, Abul Kalam Muhammad Rakibul Haque
    Ghosh, Pronab
    Hasan, Md. Zahid
    Jonkman, Mirjam
    De Boer, Friso
    BIOLOGY-BASEL, 2021, 10 (12):