Benchmarking weakly-supervised deep learning pipelines for whole slide classification in computational pathology

被引:105
作者
Laleh, Narmin Ghaffari [1 ]
Muti, Hannah Sophie [1 ]
Loeffler, Chiara Maria Lavinia [1 ]
Echle, Amelie [1 ]
Saldanha, Oliver Lester [1 ]
Mahmood, Faisal [2 ]
Lu, Ming Y. [2 ]
Trautwein, Christian [1 ]
Langer, Rupert [4 ]
Dislich, Bastian [3 ]
Buelow, Roman D. [5 ]
Grabsch, Heike Irmgard [6 ,7 ]
Brenner, Hermann [8 ,9 ,10 ]
Chang-Claude, Jenny [11 ,12 ]
Alwers, Elizabeth [8 ]
Brinker, Titus J. [13 ]
Khader, Firas [14 ]
Truhn, Daniel [14 ]
Gaisa, Nadine T. [5 ]
Boor, Peter [5 ]
Hoffmeister, Michael [8 ]
Schulz, Volkmar [15 ,16 ,17 ,18 ]
Kather, Jakob Nikolas [1 ,7 ,19 ]
机构
[1] Univ Hosp RWTH Aachen, Dept Med 3, Aachen, Germany
[2] Harvard Med Sch, Brigham & Womens Hosp, Dept Pathol, Boston, MA USA
[3] Univ Bern, Inst Pathol, Bern, Switzerland
[4] Johannes Kepler Univ Linz, Kepler Univ Hosp, Inst Pathol & Mol Pathol, Linz, Austria
[5] Univ Hosp RWTH Aachen, Inst Pathol, Aachen, Germany
[6] Maastricht Univ, GROW Sch Oncol & Dev Biol, Dept Pathol, Med Ctr, Maastricht, Netherlands
[7] Univ Leeds, Leeds Inst Med Res St Jamess, Div Pathol & Data Analyt, Leeds, England
[8] German Canc Res Ctr, Div Clin Epidemiol & Aging Res, Heidelberg, Germany
[9] German Canc Res Ctr, Div Prevent Oncol, Heidelberg, Germany
[10] German Canc Res Ctr, German Canc Consortium DKTK, Heidelberg, Germany
[11] German Canc Res Ctr, Div Canc Epidemiol, Heidelberg, Germany
[12] Univ Med Ctr Hamburg Eppendorf, Univ Canc Ctr Hamburg, Canc Epidemiol Grp, Hamburg, Germany
[13] German Canc Res Ctr, Digital Biomarkers Oncol Grp, Heidelberg, Germany
[14] Univ Hosp RWTH Aachen, Dept Radiol, Aachen, Germany
[15] Rhein Westfal TH Aachen, Dept Phys Mol Imaging Syst Expt Mol Imaging, Aachen, Germany
[16] Fraunhofer Inst Digital Med MEVIS, Bremen, Germany
[17] Univ Hosp Aachen, Comprehens Diagnost Ctr Aachen CDCA, Aachen, Germany
[18] Hyper Hybrid Imaging Syst GmbH, Aachen, Germany
[19] Tech Univ Dresden, Med Fac Carl Gustav Carus, Else Kroener Fresenius Ctr Digital Hlth, Dresden, Germany
基金
欧洲研究理事会;
关键词
Computational pathology; Artificial intelligence; Weakly-supervised deep learning; Vision transformers; Convolutional neural networks; Multiple-Instance Learning; COLORECTAL-CANCER; MICROSATELLITE INSTABILITY; PROSTATE-CANCER; NEURAL-NETWORK; COLONOSCOPY; PREDICTION; BIOPSIES;
D O I
10.1016/j.media.2022.102474
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Artificial intelligence (AI) can extract visual information from histopathological slides and yield biological insight and clinical biomarkers. Whole slide images are cut into thousands of tiles and classification problems are often weakly-supervised: the ground truth is only known for the slide, not for every single tile. In classical weakly-supervised analysis pipelines, all tiles inherit the slide label while in multiple-instance learning (MIL), only bags of tiles inherit the label. However, it is still unclear how these widely used but markedly different approaches perform relative to each other.We implemented and systematically compared six methods in six clinically relevant end-to-end prediction tasks using data from N = 2980 patients for training with rigorous external validation. We tested three classical weakly-supervised approaches with convolutional neural networks and vision transformers (ViT) and three MIL-based approaches with and without an additional attention module. Our results empirically demonstrate that histological tumor subtyping of renal cell carcinoma is an easy task in which all approaches achieve an area under the receiver operating curve (AUROC) of above 0.9. In contrast, we report significant performance differences for clinically relevant tasks of mutation prediction in colorectal, gastric, and bladder cancer. In these mutation prediction tasks, classical weakly-supervised workflows outperformed MIL-based weakly-supervised methods for mutation prediction, which is surprising given their simplicity. This shows that new end-to-end image analysis pipelines in computational pathology should be compared to classical weakly-supervised methods. Also, these findings motivate the development of new methods which combine the elegant assumptions of MIL with the empirically observed higher performance of classical weakly-supervised approaches. We make all source codes publicly available at https://github.com/KatherLab/HIA , allowing easy application of all methods to any similar task.(c) 2022 Elsevier B.V. All rights reserved.
引用
收藏
页数:15
相关论文
共 79 条
[11]   Deep learning identifies inflamed fat as a risk factor for lymph node metastasis in early colorectal cancer [J].
Brockmoeller, Scarlet ;
Echle, Amelie ;
Laleh, Narmin Ghaffari ;
Eiholm, Susanne ;
Malmstrom, Marie Louise ;
Kuhlmann, Tine Plato ;
Levic, Katarina ;
Grabsch, Heike Irmgard ;
West, Nicholas P. ;
Saldanha, Oliver Lester ;
Kouvidi, Katerina ;
Bono, Aurora ;
Heij, Lara R. ;
Brinker, Titus J. ;
Gogenur, Ismayil ;
Quirke, Philip ;
Kather, Jakob Nikolas .
JOURNAL OF PATHOLOGY, 2022, 256 (03) :269-281
[12]   Automated deep-learning system for Gleason grading of prostate cancer using biopsies: a diagnostic study [J].
Bulten, Wouter ;
Pinckaers, Hans ;
van Boven, Hester ;
Vink, Robert ;
de Bel, Thomas ;
van Ginneken, Bram ;
van der Laak, Jeroen ;
Hulsbergen-van de Kaa, Christina ;
Litjens, Geert .
LANCET ONCOLOGY, 2020, 21 (02) :233-241
[13]   Deep learning identifies morphological features in breast cancer predictive of cancer ERBB2 status and trastuzumab treatment efficacy [J].
Bychkov, Dmitrii ;
Linder, Nina ;
Tiulpin, Aleksei ;
Kuecuekel, Hakan ;
Lundin, Mikael ;
Nordling, Stig ;
Sihto, Harri ;
Isola, Jorma ;
Lehtimaeki, Tiina ;
Kellokumpu-Lehtinen, Pirkko-Liisa ;
von Smitten, Karl ;
Joensuu, Heikki ;
Lundin, Johan .
SCIENTIFIC REPORTS, 2021, 11 (01)
[14]   Clinical-grade computational pathology using weakly supervised deep learning on whole slide images [J].
Campanella, Gabriele ;
Hanna, Matthew G. ;
Geneslaw, Luke ;
Miraflor, Allen ;
Silva, Vitor Werneck Krauss ;
Busam, Klaus J. ;
Brogi, Edi ;
Reuter, Victor E. ;
Klimstra, David S. ;
Fuchs, Thomas J. .
NATURE MEDICINE, 2019, 25 (08) :1301-+
[15]   An annotation-free whole-slide training approach to pathological classification of lung cancer types using deep learning [J].
Chen, Chi-Long ;
Chen, Chi-Chung ;
Yu, Wei-Hsiang ;
Chen, Szu-Hua ;
Chang, Yu-Chan ;
Hsu, Tai-I ;
Hsiao, Michael ;
Yeh, Chao-Yuan ;
Chen, Cheng-Yu .
NATURE COMMUNICATIONS, 2021, 12 (01)
[16]   Deep learning links histology, molecular signatures and prognosis in cancer [J].
Coudray, Nicolas ;
Tsirigos, Aristotelis .
NATURE CANCER, 2020, 1 (08) :755-757
[17]   Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning [J].
Coudray, Nicolas ;
Ocampo, Paolo Santiago ;
Sakellaropoulos, Theodore ;
Narula, Navneet ;
Snuderl, Matija ;
Fenyo, David ;
Moreira, Andre L. ;
Razavian, Narges ;
Tsirigos, Aristotelis .
NATURE MEDICINE, 2018, 24 (10) :1559-+
[18]  
Das K, 2018, I S BIOMED IMAGING, P578, DOI 10.1109/ISBI.2018.8363642
[19]   COMPARING THE AREAS UNDER 2 OR MORE CORRELATED RECEIVER OPERATING CHARACTERISTIC CURVES - A NONPARAMETRIC APPROACH [J].
DELONG, ER ;
DELONG, DM ;
CLARKEPEARSON, DI .
BIOMETRICS, 1988, 44 (03) :837-845
[20]   Solving the multiple instance problem with axis-parallel rectangles [J].
Dietterich, TG ;
Lathrop, RH ;
LozanoPerez, T .
ARTIFICIAL INTELLIGENCE, 1997, 89 (1-2) :31-71