eOSCE stations live versus remote evaluation and scores variability

被引：5

作者：

Bouzid, Donia ^{[1
,2
,3
]}

Mullaert, Jimmy ^{[1
,2
]}

Ghazali, Aiham ^{[1
,2
]}

Ferre, Valentine Marie ^{[1
,2
,4
]}

Mentre, France ^{[1
,2
,5
,6
]}

Lemogne, Cedric ^{[6
,7
,8
]}

Ruszniewski, Philippe ^{[6
,9
]}

Faye, Albert ^{[6
,10
]}

Dinh, Alexy Tran ^{[6
,11
]}

Mirault, Tristan ^{[6
,12
,13
]}

机构：

[1] Univ Paris Cite, F-75018 Paris, France

[2] Univ Sorbonne Paris Nord, Inserm IAME, F-75018 Paris, France

[3] Bichat Claude Bernard Uni Hosp, AP HP, Emergency Dept, Paris, France

[4] Bichat Claude Bernard Univ Hosp, AP HP, Paris, France

[5] Bichat Claude Bernard Univ Hosp, AP HP, Dept Epidemiol Biostat & Rech Clin, Paris, France

[6] Univ Paris Cite, UFR Med, Paris, France

[7] Univ Paris Cite, INSERM U1266, Inst Psychiat & Neurosci Paris, F-75014 Paris, France

[8] Hop Hotel Dieu, AP HP, Serv Psychiat Adulte, F-75004 Paris, France

[9] Hop Beaujon, AP HP, Serv Gastroenterol & Pancreatol, Paris, France

[10] Hop Robert Debre, AP HP, Serv Pediat Gen, Paris, France

[11] Hop Bichat Claude Bernard, AP HP, Dept Anesthesie Reanimat, Paris, France

[12] Hop Europeen Georges Pompidou, AP HP, Dept Med Vasc, Paris, France

[13] Univ Paris Cite, PARCC Team 5, INSERM U970, F-75015 Paris, France

来源：

BMC MEDICAL EDUCATION | 2022年 / 22卷 / 01期

关键词：

Interrater reliability; Remote objective structured clinical examination; Global ratings; PERFORMANCE; RELIABILITY; SITE;

D O I：

10.1186/s12909-022-03919-1

中图分类号：

G40 [教育学];

学科分类号：

040101 ; 120403 ;

摘要：

Background: Objective structured clinical examinations (OSCEs) are known to be a fair evaluation method. These recent years, the use of online OSCEs (eOSCEs) has spread. This study aimed to compare remote versus live evaluation and assess the factors associated with score variability during eOSCEs. Methods: We conducted large-scale eOSCEs at the medical school of the Universite de Paris Cite in June 2021 and recorded all the students' performances, allowing a second evaluation. To assess the agreement in our context of multiple raters and students, we fitted a linear mixed model with student and rater as random effects and the score as an explained variable. Results: One hundred seventy observations were analyzed for the first station after quality control. We retained 192 and 110 observations for the statistical analysis of the two other stations. The median score and interquartile range were 60 out of 100 (IQR 50-70), 60 out of 100 (IQR 54-70), and 53 out of 100 (IQR 45-62) for the three stations. The score variance proportions explained by the rater (ICC rater) were 23.0, 16.8, and 32.8%, respectively. Of the 31 raters, 18 (58%) were male. Scores did not differ significantly according to the gender of the rater (p = 0.96, 0.10, and 0.26, respectively). The two evaluations showed no systematic difference in scores (p = 0.92, 0.053, and 0.38, respectively). Conclusion: Our study suggests that remote evaluation is as reliable as live evaluation for eOSCEs.

引用

页数：6

共 23 条

[1] A novel approach to medical school examinations during the COVID-19 pandemic [J].