The need to separate the wheat from the chaff in medical informatics Introducing a comprehensive checklist for the (self)-assessment of medical AI studies

被引:161
作者
Cabitza, Federico [1 ]
Campagner, Andrea [1 ]
机构
[1] Univ Milano Bicocca, DISCO, Viale Sarca 336, I-20126 Milan, Italy
关键词
Medical artificial intelligence; Machine learning; Checklist; Quality auditing; ARTIFICIAL-INTELLIGENCE; EXTERNAL VALIDATION; BIG DATA; PERFORMANCE; GUIDELINES; PROMISE; MODEL;
D O I
10.1016/j.ijmedinf.2021.104510
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
This editorial aims to contribute to the current debate about the quality of studies that apply machine learning (ML) methodologies to medical data to extract value from them and provide clinicians with viable and useful tools supporting everyday care practices. We propose a practical checklist to help authors to self assess the quality of their contribution and to help reviewers to recognize and appreciate high-quality medical ML studies by distinguishing them from the mere application of ML techniques to medical data.
引用
收藏
页数:7
相关论文
共 94 条
  • [1] Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis
    Aggarwal, Ravi
    Sounderajah, Viknesh
    Martin, Guy
    Ting, Daniel S. W.
    Karthikesalingam, Alan
    King, Dominic
    Ashrafian, Hutan
    Darzi, Ara
    [J]. NPJ DIGITAL MEDICINE, 2021, 4 (01)
  • [2] [Anonymous], 1975, DELPHI METHOD TECHNI
  • [3] Sample-Size Determination Methodologies for Machine Learning in Medical Imaging Research: A Systematic Review
    Balki, Indranil
    Amirabadi, Afsaneh
    Levman, Jacob
    Martel, Anne L.
    Emersic, Ziga
    Meden, Blaz
    Garcia-Pedrero, Angel
    Ramirez, Saul C.
    Kong, Dehan
    Moody, Alan R.
    Tyrrell, Pascal N.
    [J]. CANADIAN ASSOCIATION OF RADIOLOGISTS JOURNAL-JOURNAL DE L ASSOCIATION CANADIENNE DES RADIOLOGISTES, 2019, 70 (04): : 344 - 353
  • [4] Challenges to the Reproducibility of Machine Learning Models in Health Care
    Beam, Andrew L.
    Manrai, Arjun K.
    Ghassemi, Marzyeh
    [J]. JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION, 2020, 323 (04): : 305 - 306
  • [5] External validation is necessary in, prediction research: A clinical example
    Bleeker, SE
    Moll, HA
    Steyerberg, EW
    Donders, ART
    Derksen-Lubsen, G
    Grobbee, DE
    Moons, KGM
    [J]. JOURNAL OF CLINICAL EPIDEMIOLOGY, 2003, 56 (09) : 826 - 832
  • [6] Measuring the prediction error. A comparison of cross-validation, bootstrap and covariance penalty methods
    Borra, Simone
    Di Ciaccio, Agostino
    [J]. COMPUTATIONAL STATISTICS & DATA ANALYSIS, 2010, 54 (12) : 2976 - 2989
  • [7] Bouthillier X. etal, 2021, P MACH LEARN SYST, V3
  • [8] As if sand were stone. New concepts and metrics to probe the ground on which to build trustable AI
    Cabitza, Federico
    Campagner, Andrea
    Sconfienza, Luca Maria
    [J]. BMC MEDICAL INFORMATICS AND DECISION MAKING, 2020, 20 (01)
  • [9] The Elephant in the Machine: Proposing a New Metric of Data Reliability and its Application to a Medical Case to Assess Classification Reliability
    Cabitza, Federico
    Campagner, Andrea
    Albano, Domenico
    Aliprandi, Alberto
    Bruno, Alberto
    Chianca, Vito
    Corazza, Angelo
    Di Pietto, Francesco
    Gambino, Angelo
    Gitto, Salvatore
    Messina, Carmelo
    Orlandi, Davide
    Pedone, Luigi
    Zappia, Marcello
    Sconfienza, Luca Maria
    [J]. APPLIED SCIENCES-BASEL, 2020, 10 (11):
  • [10] Unintended Consequences of Machine Learning in Medicine
    Cabitza, Federico
    Rasoini, Raffaele
    Gensini, Gian Franco
    [J]. JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION, 2017, 318 (06): : 517 - 518