Multimodal biomedical AI

被引：378
作者：
Acosta, Julian N. [1 ]
Falcone, Guido J. [1 ]
Rajpurkar, Pranav [2 ]
Topol, Eric J. [3 ]
机构：
[1] Yale Sch Med, Dept Neurol, New Haven, CT USA
[2] Harvard Med Sch, Dept Biomed Informat, Boston, MA 02115 USA
[3] Scripps Res, Scripps Res Translat Inst, La Jolla, CA 92037 USA
来源：
NATURE MEDICINE | 2022年 / 28卷 / 09期
关键词：
UK BIOBANK; HEALTH-CARE; BIG DATA; CLINICAL-TRIALS; PLATFORM; FUSION; TECHNOLOGY; PREDICTION; MEDICINE; COVID-19;
D O I：
10.1038/s41591-022-01981-2
中图分类号：
Q5 [生物化学]; Q7 [分子生物学];
学科分类号：
071010 ; 081704 ;
摘要：
Multimodal artificial intelligence models could unlock many exciting applications in health and medicine; this Review outlines the most promising uses and the technical pitfalls to avoid. The increasing availability of biomedical data from large biobanks, electronic health records, medical imaging, wearable and ambient biosensors, and the lower cost of genome and microbiome sequencing have set the stage for the development of multimodal artificial intelligence solutions that capture the complexity of human health and disease. In this Review, we outline the key applications enabled, along with the technical and analytical challenges. We explore opportunities in personalized medicine, digital clinical trials, remote monitoring and care, pandemic surveillance, digital twin technology and virtual health assistants. Further, we survey the data, modeling and privacy challenges that must be overcome to realize the full potential of multimodal artificial intelligence in health.
引用
收藏
页码：1773 / 1784
页数：12
相关论文
共 177 条
  • [51] Collider bias undermines our understanding of COVID-19 disease risk and severity
    Griffith, Gareth J.
    Morris, Tim T.
    Tudball, Matthew J.
    Herbert, Annie
    Mancano, Giulia
    Pike, Lindsey
    Sharp, Gemma C.
    Sterne, Jonathan
    Palmer, Tom M.
    Smith, George Davey
    Tilling, Kate
    Zuccolo, Luisa
    Davies, Neil M.
    Hemani, Gibran
    [J]. NATURE COMMUNICATIONS, 2020, 11 (01)
  • [52] Guu Kelvin., 2020, PROCEEDINGS OF THE 3, V119, P3929
  • [53] The sequences of 150,119 genomes in the UK Biobank
    Halldorsson, Bjarni, V
    Eggertsson, Hannes P.
    Moore, Kristjan H. S.
    Hauswedell, Hannes
    Eiriksson, Ogmundur
    Ulfarsson, Magnus O.
    Palsson, Gunnar
    Hardarson, Marteinn T.
    Oddsson, Asmundur
    Jensson, Brynjar O.
    Kristmundsdottir, Snaedis
    Sigurpalsdottir, Brynja D.
    Stefansson, Olafur A.
    Beyter, Doruk
    Holley, Guillaume
    Tragante, Vinicius
    Gylfason, Arnaldur
    Olason, Pall, I
    Zink, Florian
    Asgeirsdottir, Margret
    Sverrisson, Sverrir T.
    Sigurdsson, Brynjar
    Gudjonsson, Sigurjon A.
    Sigurdsson, Gunnar T.
    Halldorsson, Gisli H.
    Sveinbjornsson, Gardar
    Norland, Kristjan
    Styrkarsdottir, Unnur
    Magnusdottir, Droplaug N.
    Snorradottir, Steinunn
    Kristinsson, Kari
    Sobech, Emilia
    Jonsson, Helgi
    Geirsson, Arni J.
    Olafsson, Isleifur
    Jonsson, Palmi
    Pedersen, Ole Birger
    Erikstrup, Christian
    Brunak, Soren
    Ostrowski, Sisse Rye
    Thorleifsson, Gudmar
    Jonsson, Frosti
    Melsted, Pall
    Jonsdottir, Ingileif
    Rafnar, Thorunn
    Holm, Hilma
    Stefansson, Hreinn
    Saemundsdottir, Jona
    Gudbjartsson, Daniel F.
    Magnusson, Olafur T.
    [J]. NATURE, 2022, 607 (7920) : 732 - +
  • [54] Assessing Missing Data Assumptions in EHR-Based Studies: A Complex and Underappreciated Task
    Haneuse, Sebastien
    Arterburn, David
    Daniels, Michael J.
    [J]. JAMA NETWORK OPEN, 2021, 4 (02)
  • [55] Illuminating the dark spaces of healthcare with ambient intelligence
    Haque, Albert
    Milstein, Arnold
    Li Fei-Fei
    [J]. NATURE, 2020, 585 (7824) : 193 - 202
  • [56] Integrating spatial gene expression and breast tumour morphology via deep learning
    He, Bryan
    Bergenstrahle, Ludvig
    Stenbeck, Linnea
    Abid, Abubakar
    Andersson, Alma
    Borg, Ake
    Maaskola, Jonas
    Lundeberg, Joakim
    Zou, James
    [J]. NATURE BIOMEDICAL ENGINEERING, 2020, 4 (08) : 827 - 834
  • [57] Current and future perspectives of liquid biopsies in genomics-driven oncology
    Heitzer, Ellen
    Haque, Imran S.
    Roberts, Charles E. S.
    Speicher, Michael R.
    [J]. NATURE REVIEWS GENETICS, 2019, 20 (02) : 71 - 88
  • [58] Decoupling the Role of Data, Attention, and Losses in Multimodal Transformers
    Hendricks, Lisa Anne
    Mellor, John
    Schneider, Rosalia
    Alayrac, Jean-Baptiste
    Nematzadeh, Aida
    [J]. TRANSACTIONS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, 2021, 9 : 570 - 585
  • [59] Digital twins for predictive oncology will be a paradigm shift for precision cancer care
    Hernandez-Boussard, Tina
    Macklin, Paul
    Greenspan, Emily J.
    Gryshuk, Amy L.
    Stahlberg, Eric
    Syeda-Mahmood, Tanveer
    Shmulevich, Ilya
    [J]. NATURE MEDICINE, 2021, 27 (12) : 2065 - 2066
  • [60] Observational Health Data Sciences and Informatics (OHDSI): Opportunities for Observational Researchers
    Hripcsak, George
    Duke, Jon D.
    Shah, Nigam H.
    Reich, Christian G.
    Huser, Vojtech
    Schuemie, Martijn J.
    Suchard, Marc A.
    Park, Rae Woong
    Wong, Ian Chi Kei
    Rijnbeek, Peter R.
    van der Lei, Johan
    Pratt, Nicole
    Noren, G. Niklas
    Li, Yu-Chuan
    Stang, Paul E.
    Madigan, David
    Ryan, Patrick B.
    [J]. MEDINFO 2015: EHEALTH-ENABLED HEALTH, 2015, 216 : 574 - 578
12345678910