Assessment of Basic Thoracic Ultrasound Skills in Immersive Virtual Reality: Gathering Validity Evidence

Objective: Operator skills are essential for thoracic ultrasound (TUS) to ensure diagnostic accuracy. Immersive virtual reality (IVR) has shown potential within medical education but never for assessment of TUS skills. This study was aimed at developing an IVR test for assessing TUS skills, gathering validity evidence and establishing a pass/fail score. Methods: An expert panel developed a test based on the TUS protocol by the European Respiratory Society (ERS), including a tutorial and two clinical cases (pleural effusion and interstitial syndrome), using an IVR platform (VitaSim, Odense, Denmark). Four anterior, four lateral and six posterior zones were available for examination and decision of diagnosis. Each correct examination equaled one point. The contrasting groups' method was used to set a pass/fail score. Results: Data were collected during the 2022 ERS Congress. We included 13 novices (N, experience: 0 TUS), 22 intermediates (I, 1-50 TUS) and 11 experienced clinicians (E, >50 TUS). Cronbach's alpha was 0.86. The total mean point scores in case 1 (C1) were (N) 5.0 +/- 2.7, (I) 7.3 +/- 2.4 and (E) 8.7 +/- 1.3, and the scores in case 2 (C2) were (N) 4.5 +/- 1.8, (I) 6.7 +/- 2.3 and (E) 8.5 +/- 2.1. Significant differences were found between N and I for C1 (p = 0.007) and C2 (p = 0.02), I and E for C1 (p = 0.04) and C2 (p = 0.019) and N and E for C1 (p < 0.001) and C2 (p < 0.001). The pass/fail score was 7 points in each case. Conclusion: We established an IVR test that can distinguish between operators with different TUS skills. This enables a standardized, objective and evidence-based approach to assessment of TUS skills.