Human-interpretable image features derived from densely mapped cancer pathology slides predict diverse molecular phenotypes

被引:149
作者
Diao, James A. [1 ,2 ]
Wang, Jason K. [1 ,2 ]
Chui, Wan Fung [1 ,2 ]
Mountain, Victoria [1 ]
Gullapally, Sai Chowdary [1 ]
Srinivasan, Ramprakash [1 ]
Mitchell, Richard N. [2 ,3 ]
Glass, Benjamin [1 ]
Hoffman, Sara [1 ]
Rao, Sudha K. [1 ]
Maheshwari, Chirag [1 ]
Lahiri, Abhik [1 ]
Prakash, Aaditya [1 ]
McLoughlin, Ryan [1 ]
Kerner, Jennifer K. [1 ]
Resnick, Murray B. [1 ,4 ]
Montalto, Michael C. [1 ]
Khosla, Aditya [1 ]
Wapinski, Ilan N. [1 ]
Beck, Andrew H. [1 ]
Elliott, Hunter L. [1 ]
Taylor-Weiner, Amaro [1 ]
机构
[1] PathAI Inc, Boston, MA 02215 USA
[2] Harvard Med Sch, Program Hlth Sci & Technol, Boston, MA 02115 USA
[3] Harvard Med Sch, Brigham & Womens Hosp, Dept Pathol, Boston, MA 02115 USA
[4] Warren Alpert Med Sch, Dept Pathol, Providence, RI USA
关键词
PD-L1; EXPRESSION; B-CELLS; IMMUNOTHERAPY; FIBROBLASTS; MACROPHAGES; SURVIVAL; GUIDE;
D O I
10.1038/s41467-021-21896-9
中图分类号
O [数理科学和化学]; P [天文学、地球科学]; Q [生物科学]; N [自然科学总论];
学科分类号
07 ; 0710 ; 09 ;
摘要
Computational methods have made substantial progress in improving the accuracy and throughput of pathology workflows for diagnostic, prognostic, and genomic prediction. Still, lack of interpretability remains a significant barrier to clinical integration. We present an approach for predicting clinically-relevant molecular phenotypes from whole-slide histopathology images using human-interpretable image features (HIFs). Our method leverages >1.6 million annotations from board-certified pathologists across >5700 samples to train deep learning models for cell and tissue classification that can exhaustively map whole-slide images at two and four micron-resolution. Cell- and tissue-type model outputs are combined into 607 HIFs that quantify specific and biologically-relevant characteristics across five cancer types. We demonstrate that these HIFs correlate with well-known markers of the tumor microenvironment and can predict diverse molecular signatures (AUROC 0.601-0.864), including expression of four immune checkpoint proteins and homologous recombination deficiency, with performance comparable to 'black-box' methods. Our HIF-based approach provides a comprehensive, quantitative, and interpretable window into the composition and spatial architecture of the tumor microenvironment. Computational methods have made progress in improving classification accuracy and throughput of pathology workflows, but lack of interpretability remains a barrier to clinical integration. Here, the authors present an approach for predicting clinically-relevant molecular phenotypes from whole-slide histopathology images using human-interpretable image features.
引用
收藏
页数:15
相关论文
共 94 条
[1]   Geospatial immune variability illuminates differential evolution of lung adenocarcinoma [J].
AbdulJabbar, Khalid ;
Raza, Shan E. Ahmed ;
Rosenthal, Rachel ;
Jamal-Hanjani, Mariam ;
Veeriah, Selvaraju ;
Akarca, Ayse ;
Lund, Tom ;
Moore, David A. ;
Salgado, Roberto ;
Al Bakir, Maise ;
Zapata, Luis ;
Hiley, Crispin T. ;
Officer, Leah ;
Sereno, Marco ;
Smith, Claire Rachel ;
Loi, Sherene ;
Hackshaw, Allan ;
Marafioti, Teresa ;
Quezada, Sergio A. ;
McGranahan, Nicholas ;
Le Quesne, John ;
Swanton, Charles ;
Yuan, Yinyin .
NATURE MEDICINE, 2020, 26 (07) :1054-+
[2]   Report on computational assessment of Tumor Infiltrating Lymphocytes from the International Immuno-Oncology Biomarker Working Group [J].
Amgad, Mohamed ;
Stovgaard, Elisabeth Specht ;
Balslev, Eva ;
Thagaard, Jeppe ;
Chen, Weijie ;
Dudgeon, Sarah ;
Sharma, Ashish ;
Kerner, Jennifer K. ;
Denkert, Carsten ;
Yuan, Yinyin ;
AbdulJabbar, Khalid ;
Wienert, Stephan ;
Savas, Peter ;
Voorwerk, Leonie ;
Beck, Andrew H. ;
Madabhushi, Anant ;
Hartman, Johan ;
Sebastian, Manu M. ;
Horlings, Hugo M. ;
Hudecek, Jan ;
Ciompi, Francesco ;
Moore, David A. ;
Singh, Rajendra ;
Roblin, Elvire ;
Balancin, Marcelo Luiz ;
Mathieu, Marie-Christine ;
Lennerz, Jochen K. ;
Kirtani, Pawan ;
Chen, I-Chun ;
Braybrooke, Jeremy P. ;
Pruneri, Giancarlo ;
Demaria, Sandra ;
Adams, Sylvia ;
Schnitt, Stuart J. ;
Lakhani, Sunil R. ;
Rojo, Federico ;
Comerma, Laura ;
Badve, Sunil S. ;
Khojasteh, Mehrnoush ;
Symmans, W. Fraser ;
Sotiriou, Christos ;
Gonzalez-Ericsson, Paula ;
Pogue-Geile, Katherine L. ;
Kim, Rim S. ;
Rimm, David L. ;
Viale, Giuseppe ;
Hewitt, Stephen M. ;
Bartlett, John M. S. ;
Penault-Llorca, Frederique ;
Goel, Shom .
NPJ BREAST CANCER, 2020, 6 (01)
[3]   Joint Region and Nucleus Segmentation for Characterization of Tumor Infiltrating Lymphocytes in Breast Cancer [J].
Amgad, Mohamed ;
Sarkar, Anindya ;
Srinivas, Chukka ;
Redman, Rachel ;
Ratra, Simrath ;
Bechert, Charles J. ;
Calhoun, Benjamin C. ;
Mrazeck, Karen ;
Kurkure, Uday ;
Cooper, Lee A. D. ;
Barnes, Michael .
MEDICAL IMAGING 2019: DIGITAL PATHOLOGY, 2019, 10956
[4]   Systematic Analysis of Breast Cancer Morphology Uncovers Stromal Features Associated with Survival [J].
Beck, Andrew H. ;
Sangoi, Ankur R. ;
Leung, Samuel ;
Marinelli, Robert J. ;
Nielsen, Torsten O. ;
van de Vijver, Marc J. ;
West, Robert B. ;
van de Rijn, Matt ;
Koller, Daphne .
SCIENCE TRANSLATIONAL MEDICINE, 2011, 3 (108)
[5]   A Human-Centered Evaluation of a Deep Learning System Deployed in Clinics for the Detection of Diabetic Retinopathy [J].
Beede, Emma ;
Baylor, Elizabeth ;
Hersch, Fred ;
Iurchenko, Anna ;
Wilcox, Lauren ;
Ruamviboonsuk, Paisan ;
Vardoulakis, Laura M. .
PROCEEDINGS OF THE 2020 CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS (CHI'20), 2020,
[6]   Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer [J].
Bejnordi, Babak Ehteshami ;
Veta, Mitko ;
van Diest, Paul Johannes ;
van Ginneken, Bram ;
Karssemeijer, Nico ;
Litjens, Geert ;
van der Laak, Jeroen A. W. M. .
JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION, 2017, 318 (22) :2199-2210
[7]   CONTROLLING THE FALSE DISCOVERY RATE - A PRACTICAL AND POWERFUL APPROACH TO MULTIPLE TESTING [J].
BENJAMINI, Y ;
HOCHBERG, Y .
JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-STATISTICAL METHODOLOGY, 1995, 57 (01) :289-300
[8]   Artificial intelligence in digital pathology - new tools for diagnosis and precision oncology [J].
Bera, Kaustav ;
Schalper, Kurt A. ;
Rimm, David L. ;
Velcheti, Vamsidhar ;
Madabhushi, Anant .
NATURE REVIEWS CLINICAL ONCOLOGY, 2019, 16 (11) :703-715
[9]   Molecular landmarks of tumor hypoxia across cancer types [J].
Bhandari, Vinayak ;
Hoey, Christianne ;
Liu, Lydia Y. ;
Lalonde, Emilie ;
Ray, Jessica ;
Livingstone, Julie ;
Lesurf, Robert ;
Shiah, Yu-Jia ;
Vujcic, Tina ;
Huang, Xiaoyong ;
Espiritu, Shadrielle M. G. ;
Heisler, Lawrence E. ;
Yousif, Fouad ;
Huang, Vincent ;
Yamaguchi, Takafumi N. ;
Yao, Cindy Q. ;
Sabelnykova, Veronica Y. ;
Fraser, Michael ;
Chua, Melvin L. K. ;
van der Kwast, Theodorus ;
Liu, Stanley K. ;
Boutros, Paul C. ;
Bristow, Robert G. .
NATURE GENETICS, 2019, 51 (02) :308-+
[10]   Patterns of TIGIT Expression in Lymphatic Tissue, Inflammation, and Cancer [J].
Blessin, Niclas C. ;
Simon, Ronald ;
Kluth, Martina ;
Fischer, Kristine ;
Hube-Magg, Claudia ;
Li, Wenchao ;
Makrypidi-Fraune, Georgia ;
Wellge, Bjoern ;
Mandelkow, Tim ;
Debatin, Nicolaus F. ;
Hoeflmayer, Doris ;
Lennartz, Maximilian ;
Sauter, Guido ;
Izbicki, Jakob R. ;
Minner, Sarah ;
Buescheck, Franziska ;
Uhlig, Ria ;
Dum, David ;
Krech, Till ;
Luebke, Andreas M. ;
Wittmer, Corinna ;
Jacobsen, Frank ;
Burandt, Eike-Christian ;
Steurer, Stefan ;
Wilczak, Waldemar ;
Hinsch, Andrea .
DISEASE MARKERS, 2019, 2019