Development of Validated Checklists to Evaluate Clinical Specialists in Pediatric ECMO Emergencies Using Delphi Method

Extracorporeal membrane oxygenation (ECMO) is a low-volume, high-risk modality of care. Clinical specialists (CS) who manage ECMO circuit emergencies vary in background and approach to circuit emergencies based on institutional training standards, leading to variation that may impact the quality of care. Validated checklists to assess CS performance are crucial to eliminate disparities and improve efficiency. This study focused on the development and validation of checklists to evaluate the clinical performance of ECMO CS in three ECMO circuit emergencies. A research team with diverse clinical background from our institution developed the first iteration of three ECMO emergency checklists: (1) venous air, (2) arterial air, and (3) oxygenator failure. A modified Delphi technique with a panel of 11 national content experts in ECMO was used to develop content validity evidence. Rating scales from 1 to 7 were used to evaluate each checklist item. The response rate for three rounds of Delphi was 100%. Items with mean score >4 were kept, and new item recommendations were added based on comments from the panel. The venous air, arterial air, and oxygenator failure checklists were revised from 10, 13, and 9 items to 12, 12, and 10 items, respectively. A Cronbach's alpha of 0.74 during the second round of responses indicated an acceptable degree of agreement. This study demonstrated content validation of three ECMO emergency checklists to assess performance of ECMO CS using a consensus-based Delphi technique. Future validity evidence should be acquired by implementing these checklists in the simulation environments.