Evolutions in clinical reasoning assessment: The Evolving Script Concordance Test

Introduction: Script concordance testing (SCT) is a method of assessment of clinical reasoning. We developed a new type of SCT case design, the evolving SCT (E-SCT), whereby the patient's clinical story is "evolving" and with thoughtful integration of new information at each stage, decisions related to clinical decision-making become increasingly clear. Objectives: We aimed to: (1) determine whether an E-SCT could differentiate clinical reasoning ability among junior residents (JR), senior residents (SR), and pediatricians, (2) evaluate the reliability of an E-SCT, and (3) obtain qualitative feedback from participants to help inform the potential acceptability of the E-SCT. Methods: A 12-case E-SCT, embedded within a 24-case pediatric SCT (PaedSCT), was administered to 91 pediatric residents (JR: n=50; SR: n=41). A total of 21 pediatricians served on the panel of experts (POE). A one-way analysis of variance (ANOVA) was conducted across the levels of experience. Participants' feedback on the E-SCT was obtained with a post-test survey and analyzed using two methods: percentage preference and thematic analysis. Results: Statistical differences existed across levels of training: F=19.31 (df=2); p < 0.001. The POE scored higher than SR (mean difference=10.34; p < 0.001) and JR (mean difference=16.00; p < 0.001). SR scored higher than JR (mean difference=5.66; p < 0.001). Reliability (Cronbach's alpha) was 0.83. Participants found the E-SCT engaging, easy to follow and true to the daily clinical decision-making process. Conclusions: The E-SCT demonstrated very good reliability and was effective in distinguishing clinical reasoning ability across three levels of experience. Participants found the E-SCT engaging and representative of real-life clinical reasoning and decision-making processes. We suggest that further refinement and utilization of the evolving style case will enhance SCT as a robust, engaging, and relevant method for the assessment of clinical reasoning.