Feasibility study of cumulative sum (CUSUM) analysis as a competency assessment tool for ultrasound-guided venous access procedures

Purpose Typically, physician training programs use a self-reported case-log competency assessment that does not measure procedure success or objectively assess competency. Cumulative sum (CUSUM) analysis could provide objective assessments of competency and progress over the training period. Our study objective was to determine the feasibility of CUSUM analysis to assess competency of ultrasound-guided peripheral intravenous (USG-PIV) line placement in pediatric anesthesiology fellows. Methods We applied a CUSUM analysis to assess competency in USG-PIV placement in three consecutive pediatric anesthesia fellowship classes. After a didactic and hands-on training session, fellows placed a USG-PIV line and self-reported the outcomes via a web-based application. Fellows were deemed competent if the CUSUM curve crossed two consecutive boundary lines from above. Results Twenty-nine fellows reported 1,388 USG-PIV attempts, with 74% success. Most fellows (26/29; 90%) achieved competency by the end of the fellowship. Two fellows identified in the mid-year CUSUM as not progressing towards competency achieved competency after targeted interventions. Fellows achieving competency (11/29 vs 25/29; odds ratio, 15; 95% confidence interval [CI], 1.98 to 113.56; P = 0.01) and attempts needed to achieve competency (19 vs 11; mean difference, 8.5; 95% CI, 3.3 to 13.7; P = 0.002) were significantly lower in younger patients and significantly higher in older patients. Conclusion Our study showed that CUSUM can be used to assess competence in USG-PIV procedures performed by pediatric anesthesia fellows and to identify learners not progressing toward competency in procedural skills. Ultrasound-guided venous access required more attempts to achieve competency in younger patients.