Multimodal biomedical AI

被引：378
作者：
Acosta, Julian N. [1 ]
Falcone, Guido J. [1 ]
Rajpurkar, Pranav [2 ]
Topol, Eric J. [3 ]
机构：
[1] Yale Sch Med, Dept Neurol, New Haven, CT USA
[2] Harvard Med Sch, Dept Biomed Informat, Boston, MA 02115 USA
[3] Scripps Res, Scripps Res Translat Inst, La Jolla, CA 92037 USA
来源：
NATURE MEDICINE | 2022年 / 28卷 / 09期
关键词：
UK BIOBANK; HEALTH-CARE; BIG DATA; CLINICAL-TRIALS; PLATFORM; FUSION; TECHNOLOGY; PREDICTION; MEDICINE; COVID-19;
D O I：
10.1038/s41591-022-01981-2
中图分类号：
Q5 [生物化学]; Q7 [分子生物学];
学科分类号：
071010 ; 081704 ;
摘要：
Multimodal artificial intelligence models could unlock many exciting applications in health and medicine; this Review outlines the most promising uses and the technical pitfalls to avoid. The increasing availability of biomedical data from large biobanks, electronic health records, medical imaging, wearable and ambient biosensors, and the lower cost of genome and microbiome sequencing have set the stage for the development of multimodal artificial intelligence solutions that capture the complexity of human health and disease. In this Review, we outline the key applications enabled, along with the technical and analytical challenges. We explore opportunities in personalized medicine, digital clinical trials, remote monitoring and care, pandemic surveillance, digital twin technology and virtual health assistants. Further, we survey the data, modeling and privacy challenges that must be overcome to realize the full potential of multimodal artificial intelligence in health.
引用
收藏
页码：1773 / 1784
页数：12
相关论文
共 177 条
  • [81] Multimodal Machine Learning Workflows for Prediction of Psychosis in Patients With Clinical High-Risk Syndromes and Recent-Onset Depression
    Koutsouleris, Nikolaos
    Dwyer, Dominic B.
    Degenhardt, Franziska
    Maj, Carlo
    Urquijo-Castro, Maria Fernanda
    Sanfelici, Rachele
    Popovic, David
    Oeztuerk, Oemer
    Haas, Shalaila S.
    Weiske, Johanna
    Ruef, Anne
    Kambeitz-Ilankovic, Lana
    Antonucci, Linda A.
    Neufang, Susanne
    Schmidt-Kraepelin, Christian
    Ruhrmann, Stephan
    Penzel, Nora
    Kambeitz, Joseph
    Haidl, Theresa K.
    Rosen, Marlene
    Chisholm, Katharine
    Riecher-Rossler, Anita
    Egloff, Laura
    Schmidt, Andre
    Andreou, Christina
    Hietala, Jarmo
    Schirmer, Timo
    Romer, Georg
    Walger, Petra
    Franscini, Maurizia
    Traber-Walker, Nina
    Schimmelmann, Benno G.
    Fluckiger, Rahel
    Michel, Chantal
    Rossler, Wulf
    Borisov, Oleg
    Krawitz, Peter M.
    Heekeren, Karsten
    Buechler, Roman
    Pantelis, Christos
    Falkai, Peter
    Salokangas, Raimo K. R.
    Lencer, Rebekka
    Bertolino, Alessandro
    Borgwardt, Stefan
    Noethen, Markus
    Brambilla, Paolo
    Wood, Stephen J.
    Upthegrove, Rachel
    Schultze-Lutter, Frauke
    [J]. JAMA PSYCHIATRY, 2021, 78 (02) : 195 - 209
  • [82] Multiscale PHATE identifies multimodal signatures of COVID-19
    Kuchroo, Manik
    Huang, Jessie
    Wong, Patrick
    Grenier, Jean-Christophe
    Shung, Dennis
    Tong, Alexander
    Lucas, Carolina
    Klein, Jon
    Burkhardt, Daniel B.
    Gigante, Scott
    Godavarthi, Abhinav
    Rieck, Bastian
    Israelow, Benjamin
    Simonov, Michael
    Mao, Tianyang
    Oh, Ji Eun
    Silva, Julio
    Takahashi, Takehiro
    Odio, Camila D.
    Casanovas-Massana, Arnau
    Farhadian, Shelli
    Dela Cruz, Charles S.
    Ko, Albert I.
    Hirn, Matthew J.
    Wilson, F. Perry
    Hussin, Julie G.
    Wolf, Guy
    Iwasaki, Akiko
    Krishnaswamy, Smita
    [J]. NATURE BIOTECHNOLOGY, 2022, 40 (05) : 681 - +
  • [83] Human fall detection on embedded platform using depth maps and wireless accelerometer
    Kwolek, Bogdan
    Kepski, Michal
    [J]. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE, 2014, 117 (03) : 489 - 501
  • [84] Li J., PREPRINT
  • [85] Imputation of missing values for electronic health record laboratory data
    Li, Jiang
    Yan, Xiaowei S.
    Chaudhary, Durgesh
    Avula, Venkatesh
    Mudiganti, Satish
    Husby, Hannah
    Shahjouei, Shima
    Afshar, Ardavan
    Stewart, Walter F.
    Yeasin, Mohammed
    Zand, Ramin
    Abedi, Vida
    [J]. NPJ DIGITAL MEDICINE, 2021, 4 (01)
  • [86] Li X., 2020, P 16 EUROPEAN C COMP
  • [87] Patient data ownership: who owns your health?
    Liddell, Kathleen
    Simon, David A.
    Lucassen, Anneke
    [J]. JOURNAL OF LAW AND THE BIOSCIENCES, 2021, 8 (02):
  • [88] Temporal Fusion Transformers for interpretable multi-horizon time series forecasting
    Lim, Bryan
    Arik, Sercan O.
    Loeff, Nicolas
    Pfister, Tomas
    [J]. INTERNATIONAL JOURNAL OF FORECASTING, 2021, 37 (04) : 1748 - 1764
  • [89] The UK Biobank imaging enhancement of 100,000 participants:rationale, data collection, management and future directions
    Littlejohns, Thomas J.
    Holliday, Jo
    Gibson, Lorna M.
    Garratt, Steve
    Oesingmann, Niels
    Alfaro-Almagro, Fidel
    Bell, Jimmy D.
    Boultwood, Chris
    Collins, Rory
    Conroy, Megan C.
    Crabtree, Nicola
    Doherty, Nicola
    Frangi, Alejandro F.
    Harvey, Nicholas C.
    Leeson, Paul
    Miller, Karla L.
    Neubauer, Stefan
    Petersen, Steffen E.
    Sellors, Jonathan
    Sheard, Simon
    Smith, Stephen M.
    Sudlow, Cathie L. M.
    Matthews, Paul M.
    Allen, Naomi E.
    [J]. NATURE COMMUNICATIONS, 2020, 11 (01)
  • [90] Disparity of Race Reporting and Representation in Clinical Trials Leading to Cancer Drug Approvals From 2008 to 2018
    Loree, Jonathan M.
    Anand, Seerat
    Dasari, Arvind
    Unger, Joseph M.
    Gothwal, Anirudh
    Ellis, Lee M.
    Varadhachary, Gauri
    Kopetz, Scott
    Overman, Michael J.
    Raghav, Kanwal
    [J]. JAMA ONCOLOGY, 2019, 5 (10)
45678910111213