Towards reproducible radiomics research: introduction of a database for radiomics studies

被引:23
作者
D'Antonoli, Tugba Akinci [1 ]
Cuocolo, Renato [2 ]
Baessler, Bettina [3 ]
dos Santos, Daniel Pinto [4 ,5 ]
机构
[1] Cantonal Hosp Baselland, Inst Radiol & Nucl Med, Liestal, Switzerland
[2] Univ Salerno, Dept Med Surg & Dent, Baronissi, Italy
[3] Univ Hosp Wurzburg, Dept Diagnost & Intervent Radiol, Wurzburg, Germany
[4] Univ Hosp Cologne, Dept Radiol, Cologne, Germany
[5] Univ Hosp Frankfurt, Dept Radiol, Frankfurt, Germany
关键词
Multiomics; Radiomics; Artificial intelligence; Reproducibility of results; IMAGES;
D O I
10.1007/s00330-023-10095-3
中图分类号
R8 [特种医学]; R445 [影像诊断学];
学科分类号
1002 ; 100207 ; 1009 ;
摘要
Objectives To investigate the model-, code-, and data-sharing practices in the current radiomics research landscape and to introduce a radiomics research database.Methods A total of 1254 articles published between January 1, 2021, and December 31, 2022, in leading radiology journals (European Radiology, European Journal of Radiology, Radiology, Radiology: Artificial Intelligence, Radiology: Cardiotho-racic Imaging, Radiology: Imaging Cancer) were retrospectively screened, and 257 original research articles were included in this study. The categorical variables were compared using Fisher's exact tests or chi-square test and numerical variables using Student's t test with relation to the year of publication.Results Half of the articles (128 of 257) shared the model by either including the final model formula or reporting the coefficients of selected radiomics features. A total of 73 (28%) models were validated on an external independent dataset. Only 16 (6%) articles shared the data or used publicly available open datasets. Similarly, only 20 (7%) of the articles shared the code. A total of 7 (3%) articles both shared code and data. All collected data in this study is presented in a radiomics research database (RadBase) and could be accessed at https://github.com/EuSoMII/RadBase.Conclusion According to the results of this study, the majority of published radiomics models were not technically repro-ducible since they shared neither model nor code and data. There is still room for improvement in carrying out reproducible and open research in the field of radiomics.Clinical relevance statement To date, the reproducibility of radiomics research and open science practices within the radiom-ics research community are still very low. Ensuring reproducible radiomics research with model-, code-, and data-shar ing practices will facilitate faster clinical translation.
引用
收藏
页码:436 / 443
页数:8
相关论文
共 25 条
[1]   Challenges to the Reproducibility of Machine Learning Models in Health Care [J].
Beam, Andrew L. ;
Manrai, Arjun K. ;
Ghassemi, Marzyeh .
JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION, 2020, 323 (04) :305-306
[2]   MAIC-10 brief quality checklist for publications using artificial intelligence and medical images [J].
Cerda-Alberich, Leonor ;
Solana, Jimena ;
Mallol, Pedro ;
Ribas, Gloria ;
Garcia-Junco, Miguel ;
Alberich-Bayarri, Angel ;
Marti-Bonmati, Luis .
INSIGHTS INTO IMAGING, 2023, 14 (01)
[3]   Radiomics: Images Are More than Pictures, They Are Data [J].
Gillies, Robert J. ;
Kinahan, Paul E. ;
Hricak, Hedvig .
RADIOLOGY, 2016, 278 (02) :563-577
[4]   What does research reproducibility mean? [J].
Goodman, Steven N. ;
Fanelli, Daniele ;
Ioannidis, John P. A. .
SCIENCE TRANSLATIONAL MEDICINE, 2016, 8 (341)
[5]   Reproducibility standards for machine learning in the life sciences [J].
Heil, Benjamin J. ;
Hoffman, Michael M. ;
Markowetz, Florian ;
Lee, Su-In ;
Greene, Casey S. ;
Hicks, Stephanie C. .
NATURE METHODS, 2021, 18 (10) :1132-1135
[6]   Open science, the replication crisis, and environmental public health [J].
Hicks, Daniel J. .
ACCOUNTABILITY IN RESEARCH-POLICIES AND QUALITY ASSURANCE, 2023, 30 (01) :34-62
[7]   Reproducible Artificial Intelligence Research Requires Open Communication of Complete Source Code [J].
Kitamura, Felipe C. ;
Pan, Ian ;
Kline, Timothy L. .
RADIOLOGY-ARTIFICIAL INTELLIGENCE, 2020, 2 (04)
[8]   CheckList for EvaluAtion of Radiomics research (CLEAR): a step-by-step reporting guideline for authors and reviewers endorsed by ESR and EuSoMII [J].
Kocak, Burak ;
Baessler, Bettina ;
Bakas, Spyridon ;
Cuocolo, Renato ;
Fedorov, Andrey ;
Maier-Hein, Lena ;
Mercaldo, Nathaniel ;
Mueller, Henning ;
Orlhac, Fanny ;
Pinto dos Santos, Daniel ;
Stanzione, Arnaldo ;
Ugga, Lorenzo ;
Zwanenburg, Alex .
INSIGHTS INTO IMAGING, 2023, 14 (01)
[9]   Radiomics: Extracting more information from medical images using advanced feature analysis [J].
Lambin, Philippe ;
Rios-Velazquez, Emmanuel ;
Leijenaar, Ralph ;
Carvalho, Sara ;
van Stiphout, Ruud G. P. M. ;
Granton, Patrick ;
Zegers, Catharina M. L. ;
Gillies, Robert ;
Boellard, Ronald ;
Dekker, Andre ;
Aerts, Hugo J. W. L. .
EUROPEAN JOURNAL OF CANCER, 2012, 48 (04) :441-446
[10]   A radiomics approach for lung nodule detection in thoracic CT images based on the dynamic patterns of morphological variation [J].
Lin, Fan-Ya ;
Chang, Yeun-Chung ;
Huang, Hsuan-Yu ;
Li, Chia-Chen ;
Chen, Yi-Chang ;
Chen, Chung-Ming .
EUROPEAN RADIOLOGY, 2022, 32 (06) :3767-3777