Prognostic models of in-hospital mortality of intensive care patients using neural representation of unstructured text: A systematic review and critical appraisal

被引:2
|
作者
Vagliano, I. [1 ,2 ]
Dormosh, N. [1 ,2 ]
Rios, M. [3 ]
Luik, T. T. [2 ,4 ]
Buonocore, T. M. [5 ]
Elbers, P. W. G. [2 ,6 ]
Dongelmans, D. A. [2 ,7 ,8 ]
Schut, M. C. [1 ,2 ,9 ]
Abu-Hanna, A. [1 ,2 ]
机构
[1] Univ Amsterdam, Dept Med Informat, Amsterdam UMC, Meibergdreef 9, NL-1105 AZ Amsterdam, Netherlands
[2] Amsterdam Publ Hlth APH, Amsterdam, Netherlands
[3] Univ Vienna, Ctr Translat Studies, Vienna, Austria
[4] Univ Amsterdam, Dept Med Biol, Amsterdam UMC, Amsterdam, Netherlands
[5] Univ Pavia, Dept Elect Comp & Biomed Engn, Pavia, Italy
[6] Vrije Univ Amsterdam, Amsterdam Inst Infect & Immun AII, Amsterdam Med Data Sci AMDS, Amsterdam UMC,Dept Intens Care Med,Ctr Crit Care C, Amsterdam, Netherlands
[7] Natl Intens Care Evaluat NICE Fdn, Amsterdam, Netherlands
[8] Univ Amsterdam, Dept Intens Care Med, Amsterdam UMC, Amsterdam, Netherlands
[9] Vrije Univ Amsterdam, Dept Clin Chem, Amsterdam UMC, Amsterdam, Netherlands
关键词
Mortality; Intensive care; Prognostic models; Natural language processing; Machine learning; Systematic review; PREDICTION MODEL; RESUSCITATE; CHALLENGES; RECORDS; RISK;
D O I
10.1016/j.jbi.2023.104504
中图分类号
TP39 [计算机的应用];
学科分类号
081203 ; 0835 ;
摘要
Objective: To review and critically appraise published and preprint reports of prognostic models of in-hospital mortality of patients in the intensive-care unit (ICU) based on neural representations (embeddings) of clinical notes. Methods: PubMed and arXiv were searched up to August 1, 2022. At least two reviewers independently selected the studies that developed a prognostic model of in-hospital mortality of intensive-care patients using free-text represented as embeddings and extracted data using the CHARMS checklist. Risk of bias was assessed using PROBAST. Reporting on the model was assessed with the TRIPOD guideline. To assess the machine learning components that were used in the models, we present a new descriptive framework based on different techniques to represent text and provide predictions from text. The study protocol was registered in the PROSPERO database (CRD42022354602). Results: Eighteen studies out of 2,825 were included. All studies used the publicly-available MIMIC dataset. Context-independent word embeddings are widely used. Model discrimination was provided by all studies (AUROC 0.75-0.96), but measures of calibration were scarce. Seven studies used both structural clinical vari-ables and notes. Model discrimination improved when adding clinical notes to variables. None of the models was externally validated and often a simple train/test split was used for internal validation. Our critical appraisal demonstrated a high risk of bias in all studies and concerns regarding their applicability in clinical practice. Conclusion: All studies used a neural architecture for prediction and were based on one publicly available dataset. Clinical notes were reported to improve predictive performance when used in addition to only clinical variables. Most studies had methodological, reporting, and applicability issues. We recommend reporting both model discrimination and calibration, using additional data sources, and using more robust evaluation strategies, including prospective and external validation. Finally, sharing data and code is encouraged to improve study reproducibility.
引用
收藏
页数:11
相关论文
共 50 条
  • [1] Use of unstructured text in prognostic clinical prediction models: a systematic review
    Seinen, Tom M.
    Fridgeirsson, Egill A.
    Ioannou, Solomon
    Jeannetot, Daniel
    John, Luis H.
    Kors, Jan A.
    Markus, Aniek F.
    Pera, Victor
    Rekkas, Alexandros
    Williams, Ross D.
    Yang, Cynthia
    van Mulligen, Erik M.
    Rijnbeek, Peter R.
    JOURNAL OF THE AMERICAN MEDICAL INFORMATICS ASSOCIATION, 2022, : 1292 - 1302
  • [2] Out-of-hours discharge from intensive care, in-hospital mortality and intensive care readmission rates: A systematic review protocol
    Vollam S.A.
    Dutton S.J.
    Young D.
    Watkinson P.J.
    Systematic Reviews, 4 (1)
  • [3] Serum Lactate as an Independent Predictor of In-Hospital Mortality in Intensive Care Patients
    Chebl, Ralphe Bou
    Tamim, Hani
    Dagher, Gilbert Abou
    Sadat, Musharaf
    Enezi, Farhan Al
    Arabi, Yaseen M.
    JOURNAL OF INTENSIVE CARE MEDICINE, 2020, 35 (11) : 1257 - 1264
  • [4] An in-hospital mortality equation for mechanically ventilated patients in intensive care units
    Umegaki, Takeshi
    Nishimura, Masaji
    Tajimi, Kimitaka
    Fushimi, Kiyohide
    Ikai, Hiroshi
    Imanaka, Yuichi
    JOURNAL OF ANESTHESIA, 2013, 27 (04) : 541 - 549
  • [5] Out-of-hours discharge from intensive care, in-hospital mortality and intensive care readmission rates: a systematic review and meta-analysis
    Vollam, Sarah
    Dutton, Susan
    Lamb, Sallie
    Petrinic, Tatjana
    Young, J. Duncan
    Watkinson, Peter
    INTENSIVE CARE MEDICINE, 2018, 44 (07) : 1115 - 1129
  • [6] Reporting and Methods in Developing Prognostic Prediction Models for Metabolic Syndrome: A Systematic Review and Critical Appraisal
    Zhang, Hui
    Shao, Jing
    Chen, Dandan
    Zou, Ping
    Cui, Nianqi
    Tang, Leiwen
    Wang, Dan
    Ye, Zhihong
    DIABETES METABOLIC SYNDROME AND OBESITY, 2020, 13 : 4981 - 4992
  • [7] Prognostic models for predicting in-hospital paediatric mortality in resource-limited countries: a systematic review
    Ogero, Morris
    Sarguta, Rachel Jelagat
    Malla, Lucas
    Aluvaala, Jalemba
    Agweyu, Ambrose
    English, Mike
    Onyango, Nelson Owuor
    Akech, Samuel
    BMJ OPEN, 2020, 10 (10):
  • [8] Prognostic models for outcome prediction following in-hospital cardiac arrest using pre-arrest factors: a systematic review, meta-analysis and critical appraisal
    van Ravenhorst, Casey Grandbois
    Schluep, Marc
    Endeman, Henrik
    Stolker, Robert-Jan
    Hoeks, Sanne Elisabeth
    CRITICAL CARE, 2023, 27 (01)
  • [9] Predicting in-hospital mortality of patients with febrile neutropenia using machine learning models
    Du, Xinsong
    Min, Jae
    Shah, Chintan P.
    Bishnoi, Rohit
    Hogan, William R.
    Lemas, Dominick J.
    INTERNATIONAL JOURNAL OF MEDICAL INFORMATICS, 2020, 139
  • [10] PROGNOSTIC ACCURACY OF MACHINE LEARNING MODELS FOR IN-HOSPITAL MORTALITY AMONG CHILDREN WITH PHOENIX SEPSIS ADMITTED TO THE PEDIATRIC INTENSIVE CARE UNIT
    Moore, Ronald
    Chanci, Daniela
    Brown, Stephanie
    Ripple, Michael J.
    Bishop, Natalie R.
    Grunwell, Jocelyn
    Kamaleswaran, Rishikesan
    SHOCK, 2025, 63 (01): : 80 - 87