Multimodal biomedical AI

被引：378
作者：
Acosta, Julian N. [1 ]
Falcone, Guido J. [1 ]
Rajpurkar, Pranav [2 ]
Topol, Eric J. [3 ]
机构：
[1] Yale Sch Med, Dept Neurol, New Haven, CT USA
[2] Harvard Med Sch, Dept Biomed Informat, Boston, MA 02115 USA
[3] Scripps Res, Scripps Res Translat Inst, La Jolla, CA 92037 USA
来源：
NATURE MEDICINE | 2022年 / 28卷 / 09期
关键词：
UK BIOBANK; HEALTH-CARE; BIG DATA; CLINICAL-TRIALS; PLATFORM; FUSION; TECHNOLOGY; PREDICTION; MEDICINE; COVID-19;
D O I：
10.1038/s41591-022-01981-2
中图分类号：
Q5 [生物化学]; Q7 [分子生物学];
学科分类号：
071010 ; 081704 ;
摘要：
Multimodal artificial intelligence models could unlock many exciting applications in health and medicine; this Review outlines the most promising uses and the technical pitfalls to avoid. The increasing availability of biomedical data from large biobanks, electronic health records, medical imaging, wearable and ambient biosensors, and the lower cost of genome and microbiome sequencing have set the stage for the development of multimodal artificial intelligence solutions that capture the complexity of human health and disease. In this Review, we outline the key applications enabled, along with the technical and analytical challenges. We explore opportunities in personalized medicine, digital clinical trials, remote monitoring and care, pandemic surveillance, digital twin technology and virtual health assistants. Further, we survey the data, modeling and privacy challenges that must be overcome to realize the full potential of multimodal artificial intelligence in health.
引用
收藏
页码：1773 / 1784
页数：12
相关论文
共 177 条
  • [1] AbbVie, 2019, SYNTH CONTR ARM END
  • [2] North American Prodrome Longitudinal Study (NAPLS 2): Overview and recruitment
    Addington, Jean
    Cadenhead, Kristin S.
    Cornblatt, Barbara A.
    Mathalon, Daniel H.
    McGlashan, Thomas H.
    Perkins, Diana O.
    Seidman, Larry J.
    Tsuang, Ming T.
    Walker, Elaine F.
    Woods, Scott W.
    Addington, Jack A.
    Cannon, Tyrone D.
    [J]. SCHIZOPHRENIA RESEARCH, 2012, 142 (1-3) : 77 - 82
  • [3] Akbari H, 2021, ADV NEUR IN
  • [4] The "All of Us" Research Program
    Denny J.C.
    Rutter J.L.
    Goldstein D.B.
    Philippakis A.
    Smoller J.W.
    Jenkins G.
    Dishman E.
    [J]. NEW ENGLAND JOURNAL OF MEDICINE, 2019, 381 (07) : 668 - 676
  • [5] A global reference for human genetic variation
    Altshuler, David M.
    Durbin, Richard M.
    Abecasis, Goncalo R.
    Bentley, David R.
    Chakravarti, Aravinda
    Clark, Andrew G.
    Donnelly, Peter
    Eichler, Evan E.
    Flicek, Paul
    Gabriel, Stacey B.
    Gibbs, Richard A.
    Green, Eric D.
    Hurles, Matthew E.
    Knoppers, Bartha M.
    Korbel, Jan O.
    Lander, Eric S.
    Lee, Charles
    Lehrach, Hans
    Mardis, Elaine R.
    Marth, Gabor T.
    McVean, Gil A.
    Nickerson, Deborah A.
    Wang, Jun
    Wilson, Richard K.
    Boerwinkle, Eric
    Doddapaneni, Harsha
    Han, Yi
    Korchina, Viktoriya
    Kovar, Christie
    Lee, Sandra
    Muzny, Donna
    Reid, Jeffrey G.
    Zhu, Yiming
    Chang, Yuqi
    Feng, Qiang
    Fang, Xiaodong
    Guo, Xiaosen
    Jian, Min
    Jiang, Hui
    Jin, Xin
    Lan, Tianming
    Li, Guoqing
    Li, Jingxiang
    Li, Yingrui
    Liu, Shengmao
    Liu, Xiao
    Lu, Yao
    Ma, Xuedi
    Tang, Meifang
    Wang, Bo
    [J]. NATURE, 2015, 526 (7571) : 68 - +
  • [6] The Project Baseline Health Study: a step towards a broader mission to map human health
    Arges, Kristine
    Assimes, Themistocles
    Bajaj, Vikram
    Balu, Suresh
    Bashir, Mustafa R.
    Beskow, Laura
    Blanco, Rosalia
    Califf, Robert
    Campbell, Paul
    Carin, Larry
    Christian, Victoria
    Cousins, Scott
    Das, Millie
    Dockery, Marie
    Douglas, Pamela S.
    Dunham, Ashley
    Eckstrand, Julie
    Fleischmann, Dominik
    Ford, Emily
    Fraulo, Elizabeth
    French, John
    Gambhir, Sanjiv S.
    Ginsburg, Geoffrey S.
    Green, Robert C.
    Haddad, Francois
    Hernandez, Adrian
    Hernandez, John
    Huang, Erich S.
    Jaffe, Glenn
    King, Daniel
    Koweek, Lynne H.
    Langlotz, Curtis
    Liao, Yaping J.
    Mahaffey, Kenneth W.
    Marcom, Kelly
    Marks, William J.
    Maron, David
    McCabe, Reid
    McCall, Shannon
    McCue, Rebecca
    Mega, Jessica
    Miller, David
    Muhlbaier, Lawrence H.
    Munshi, Rajan
    Newby, L. Kristin
    Pak-Harvey, Ezra
    Patrick-Lake, Bray
    Pencina, Michael
    Peterson, Eric D.
    Rodriguez, Fatima
    [J]. NPJ DIGITAL MEDICINE, 2020, 3 (01)
  • [7] Baevski A., PREPRINT
  • [8] Advancing COVID-19 diagnosis with privacy-preserving collaboration in artificial intelligence
    Bai, Xiang
    Wang, Hanchen
    Ma, Liya
    Xu, Yongchao
    Gan, Jiefeng
    Fan, Ziwei
    Yang, Fan
    Ma, Ke
    Yang, Jiehua
    Bai, Song
    Shu, Chang
    Zou, Xinyu
    Huang, Renhao
    Zhang, Changzheng
    Liu, Xiaowu
    Tu, Dandan
    Xu, Chuou
    Zhang, Wenqing
    Wang, Xi
    Chen, Anguo
    Zeng, Yu
    Yang, Dehua
    Wang, Ming-Wei
    Holalkere, Nagaraj
    Halin, Neil J.
    Kamel, Ihab R.
    Wu, Jia
    Peng, Xuehua
    Wang, Xiang
    Shao, Jianbo
    Mongkolwat, Pattanasak
    Zhang, Jianjun
    Liu, Weiyang
    Roberts, Michael
    Teng, Zhongzhao
    Beer, Lucian
    Sanchez, Lorena E.
    Sala, Evis
    Rubin, Daniel L.
    Weller, Adrian
    Lasenby, Joan
    Zheng, Chuangsheng
    Wang, Jianming
    Li, Zhen
    Schonlieb, Carola
    Xia, Tian
    [J]. NATURE MACHINE INTELLIGENCE, 2021, 3 (12) : 1081 - 1089
  • [9] How should we think about clinical data ownership?
    Ballantyne, Angela
    [J]. JOURNAL OF MEDICAL ETHICS, 2020, 46 (05) : 289 - 294
  • [10] Multimodal Machine Learning: A Survey and Taxonomy
    Baltrusaitis, Tadas
    Ahuja, Chaitanya
    Morency, Louis-Philippe
    [J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2019, 41 (02) : 423 - 443
12345678910