Development of a machine learning-based acuity score prediction model for virtual care settings

被引:5
作者
Hall, Justin N. [1 ,2 ,3 ]
Galaev, Ron [4 ]
Gavrilov, Marina [4 ]
Mondoux, Shawn [3 ,5 ,6 ]
机构
[1] Sunnybrook Hlth Sci Ctr, Dept Emergency Serv, C753, Toronto, ON M4N 3M5, Canada
[2] Univ Toronto, Temerty Fac Med, Dept Med, Div Emergency Med, Toronto, ON, Canada
[3] Univ Toronto, Inst Hlth Policy Management & Evaluat, Toronto, ON, Canada
[4] EmergConnect, Thornhill, ON, Canada
[5] St Josephs Healthcare Hamilton, Dept Emergency Med, Hamilton, ON, Canada
[6] McMaster Univ, Dept Med, Div Emergency Med, Hamilton, ON, Canada
关键词
Machine learning; Prediction model; Acuity scores; Remote triage; Virtual care; EMERGENCY-DEPARTMENT TRIAGE; ELECTRONIC TRIAGE; CANADIAN TRIAGE; SCALE; RELIABILITY; AGREEMENT; MEDICINE; NURSES;
D O I
10.1186/s12911-023-02307-z
中图分类号
R-058 [];
学科分类号
摘要
ObjectiveHealthcare is increasingly digitized, yet remote and automated machine learning (ML) triage prediction systems for virtual urgent care use remain limited. The Canadian Triage and Acuity Scale (CTAS) is the gold standard triage tool for in-person care in Canada. The current work describes the development of a ML-based acuity score modelled after the CTAS system.MethodsThe ML-based acuity score model was developed using 2,460,109 de-identified patient-level encounter records from three large healthcare organizations (Ontario, Canada). Data included presenting complaint, clinical modifiers, age, sex, and self-reported pain. 2,041,987 records were high acuity (CTAS 1-3) and 416,870 records were low acuity (CTAS 4-5). Five models were trained: decision tree, k-nearest neighbors, random forest, gradient boosting regressor, and neural net. The outcome variable of interest was the acuity score predicted by the ML system compared to the CTAS score assigned by the triage nurse.ResultsGradient boosting regressor demonstrated the greatest prediction accuracy. This final model was tuned toward up triaging to minimize patient risk if adopted into the clinical context. The algorithm predicted the same score in 47.4% of cases, and the same or more acute score in 95.0% of cases.ConclusionsThe ML algorithm shows reasonable predictive accuracy and high predictive safety and was developed using the largest dataset of its kind to date. Future work will involve conducting a pilot study to validate and prospectively assess reliability of the ML algorithm to assign acuity scores remotely.
引用
收藏
页数:10
相关论文
共 41 条
  • [1] Ahsan Kazi Badrul, 2019, Journal of Industrial Engineering International, V15, P253, DOI 10.1007/s40092-019-00335-x
  • [2] Pediatric triage variations among nurses, pediatric and emergency residents using the Canadian triage and acuity scale
    Alshaibi, Saleh
    AlBassri, Tala
    AlQeuflie, Suliman
    Philip, Winnie
    Alharthy, Nesrin
    [J]. BMC EMERGENCY MEDICINE, 2021, 21 (01)
  • [3] Saving emergency medicine: is less more?
    Atkinson, Paul
    McGeorge, Ken
    Innes, Grant
    [J]. CANADIAN JOURNAL OF EMERGENCY MEDICINE, 2022, 24 (01) : 9 - 11
  • [4] Bestsennyy OG., 2021, TELEHEALTH QUARTER T
  • [5] Reliability of the Canadian emergency department triage and acuity scale: Interrater agreement
    Beveridge, R
    Ducharme, J
    Janes, L
    Beaulieu, S
    Walter, S
    [J]. ANNALS OF EMERGENCY MEDICINE, 1999, 34 (02) : 155 - 159
  • [6] Effectiveness of Acute Care Remote Triage Systems: a Systematic Review
    Boggan, Joel C.
    Shoup, John Paul
    Whited, John D.
    Van Voorhees, Elizabeth
    Gordon, Adelaide M.
    Rushton, Sharron
    Lewinski, Allison A.
    Tabriz, Amir A.
    Adam, Soheir
    Fulton, Jessica
    Kosinski, Andrzej S.
    Van Noord, Megan G.
    Williams, John W., Jr.
    Goldstein, Karen M.
    Gierisch, Jennifer M.
    [J]. JOURNAL OF GENERAL INTERNAL MEDICINE, 2020, 35 (07) : 2136 - 2145
  • [7] Canada Health Infoway, 2015, Report on Digital Health: Canadians embracing Digital Health
  • [8] CMA, 2020, What Canadians Think About Virtual Care
  • [9] Cotton R., 2020, Can J Emerg Nurs, V44, P19, DOI [10.29173/cjen130, DOI 10.29173/CJEN130]
  • [10] EMERGENCY DEPARTMENT TRIAGE: DO EXPERIENCED NURSES AGREE ON TRIAGE SCORES?
    Dallaire, Clemence
    Poitras, Julien
    Aubin, Karine
    Lavoie, Andre
    Moore, Lynne
    [J]. JOURNAL OF EMERGENCY MEDICINE, 2012, 42 (06) : 736 - 740