Machine learning for developing a prediction model of hospital admission of emergency department patients: Hype or hope?

被引:29
作者
De Hond, Anne [1 ,2 ,3 ]
Raven, Wouter [4 ]
Schinkelshoek, Laurens [1 ,2 ]
Gaakeer, Menno [5 ]
Ter Avest, Ewoud [6 ]
Sir, Ozcan [7 ]
Lameijer, Heleen [8 ]
Hessels, Roger Apa [9 ]
Reijnen, Resi [10 ]
De Jonge, Evert [11 ]
Steyerberg, Ewout [3 ]
Nickel, Christian H. [12 ]
De Groot, Bas [4 ]
机构
[1] Leiden Univ, Dept Informat Technol & Digital Innovat, Med Ctr, Albinusdreef 2, NL-2300 RC Leiden, Netherlands
[2] Leiden Univ, Clin AI Implementat & Res Lab, Med Ctr, Albinusdreef 2, NL-2300 RC Leiden, Netherlands
[3] Leiden Univ, Dept Biomed Data Sci, Med Ctr, Albinusdreef 2, NL-2300 RC Leiden, Netherlands
[4] Leiden Univ, Dept Emergency Med, Med Ctr, Albinusdreef 2, NL-2300 RC Leiden, Netherlands
[5] Adrz Hosp, Dept Emergency Med, S Gravenpolderseweg 114, NL-4462 RA Goes, Netherlands
[6] Univ Med Ctr Groningen, Dept Emergency Med, Hanzepl 1, NL-9713 GZ Groningen, Netherlands
[7] Radboud Univ Nijmegen, Dept Emergency Med, Med Ctr, Houtlaan 4, NL-6525 XZ Nijmegen, Netherlands
[8] Med Ctr Leeuwarden, Dept Emergency Med, Henri Dunantweg 2, NL-8934 AD Leeuwarden, Netherlands
[9] Elisabeth TweeSteden Hosp, Dept Emergency Med, Doctor Deelenlaan 5, NL-5042 AD Tilburg, Netherlands
[10] Haaglanden Med Ctr, Dept Emergency Med, Lijnbaan 32, NL-2512 VA The Hague, Netherlands
[11] Leiden Univ, Dept Intens Care Med, Med Ctr, Albinusdreef 2, NL-2300 RC Leiden, Netherlands
[12] Univ Basel, Univ Hosp Basel, Dept Emergency Med, Basel, Switzerland
关键词
Machine learning; Prognosis; Prediction; Emergency medicine; Hospital admission; Triage; REGRESSION; CARE;
D O I
10.1016/j.ijmedinf.2021.104496
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Objective: Early identification of emergency department (ED) patients who need hospitalization is essential for quality of care and patient safety. We aimed to compare machine learning (ML) models predicting the hospitalization of ED patients and conventional regression techniques at three points in time after ED registration. Methods: We analyzed consecutive ED patients of three hospitals using the Netherlands Emergency Department Evaluation Database (NEED). We developed prediction models for hospitalization using an increasing number of data available at triage, similar to 30 min (including vital signs) and similar to 2 h (including laboratory tests) after ED registration, using ML (random forest, gradient boosted decision trees, deep neural networks) and multivariable logistic regression analysis (including spline transformations for continuous predictors). Demographics, urgency, presenting complaints, disease severity and proxies for comorbidity, and complexity were used as covariates. We compared the performance using the area under the ROC curve in independent validation sets from each hospital. Results: We included 172,104 ED patients of whom 66,782 (39 %) were hospitalized. The AUC of the multi-variable logistic regression model was 0.82 (0.78-0.86) at triage, 0.84 (0.81-0.86) at similar to 30 min and 0.83 (0.75-0.92) after similar to 2 h. The best performing ML model over time was the gradient boosted decision trees model with an AUC of 0.84 (0.77-0.88) at triage, 0.86 (0.82-0.89) at similar to 30 min and 0.86 (0.74-0.93) after similar to 2 h. Conclusions: Our study showed that machine learning models had an excellent but similar predictive performance as the logistic regression model for predicting hospital admission. In comparison to the 30-min model, the 2-h model did not show a performance improvement. After further validation, these prediction models could support management decisions by real-time feedback to medical personal.
引用
收藏
页数:8
相关论文
共 34 条
[1]  
Afnan M.A.M., 2020, EMERG MED J
[2]   Progressive prediction of hospitalisation in the emergency department: uncovering hidden patterns to improve patient flow [J].
Bara-Corren, Yuval ;
Israelit, Shlomo Hanan ;
Reis, Ben Y. .
EMERGENCY MEDICINE JOURNAL, 2017, 34 (05) :308-314
[3]   The Effect of Emergency Department Crowding on Clinically Oriented Outcomes [J].
Bernstein, Steven L. ;
Aronsky, Dominik ;
Duseja, Reena ;
Epstein, Stephen ;
Handel, Dan ;
Hwang, Ula ;
McCarthy, Melissa ;
McConnell, K. John ;
Pines, Jesse M. ;
Rathlev, Niels ;
Schafermeyer, Robert ;
Zwemer, Frank ;
Schull, Michael ;
Asplin, Brent R. .
ACADEMIC EMERGENCY MEDICINE, 2009, 16 (01) :1-10
[4]   Physicians' Disease Severity Ratings are Non-Inferior to the Emergency Severity Index [J].
Bingisser, Roland ;
Baerlocher, Severin Manuel ;
Kuster, Tobias ;
Ortega, Ricardo Nieves ;
Nickel, Christian H. .
JOURNAL OF CLINICAL MEDICINE, 2020, 9 (03)
[5]   A simple tool to predict admission at the time of triage [J].
Cameron, Allan ;
Rodgers, Kenneth ;
Ireland, Alastair ;
Jamdar, Ravi ;
McKay, Gerard A. .
EMERGENCY MEDICINE JOURNAL, 2015, 32 (03) :174-179
[6]   Artificial intelligence, bias and clinical safety [J].
Challen, Robert ;
Denny, Joshua ;
Pitt, Martin ;
Gompels, Luke ;
Edwards, Tom ;
Tsaneva-Atanasova, Krasimira .
BMJ QUALITY & SAFETY, 2019, 28 (03) :231-237
[7]   XGBoost: A Scalable Tree Boosting System [J].
Chen, Tianqi ;
Guestrin, Carlos .
KDD'16: PROCEEDINGS OF THE 22ND ACM SIGKDD INTERNATIONAL CONFERENCE ON KNOWLEDGE DISCOVERY AND DATA MINING, 2016, :785-794
[8]   A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models [J].
Christodoulou, Evangelia ;
Ma, Jie ;
Collins, Gary S. ;
Steyerberg, Ewout W. ;
Verbakel, Jan Y. ;
Van Calster, Ben .
JOURNAL OF CLINICAL EPIDEMIOLOGY, 2019, 110 :12-22
[9]   Access block causes emergency department overcrowding and ambulance diversion in Perth, Western Australia [J].
Fatovich, DM ;
Nagree, Y ;
Sprivulis, P .
EMERGENCY MEDICINE JOURNAL, 2005, 22 (05) :351-354
[10]   Clinical Decision Support Systems for Triage in the Emergency Department using Intelligent Systems: a Review [J].
Fernandes, Marta ;
Vieira, Susana M. ;
Leite, Francisca ;
Palos, Carlos ;
Finkelstein, Stan ;
Sousa, Joao M. C. .
ARTIFICIAL INTELLIGENCE IN MEDICINE, 2020, 102