Evidence for the Validity of Grouped Self-Assessments in Measuring the Outcomes of Educational Programs

被引:16
作者
D'Eon, Marcel F. [1 ]
Trinder, Krista [1 ]
机构
[1] Univ Saskatchewan, Coll Med, Saskatoon, SK S7M 3L6, Canada
关键词
grouped self-assessments; outcome measures; validity; program evaluation; correlational analysis; DETECT WORKSHOP SUCCESS; MEDICAL-STUDENTS; METHODOLOGY; RESIDENTS; ACCURACY;
D O I
10.1177/0163278713475868
中图分类号
R19 [保健组织与事业(卫生事业管理)];
学科分类号
摘要
There is compelling empirical evidence in support of the use of grouped self-assessment data to measure program outcomes. However, other credible research has clearly shown that self-assessments are poor predictors of individual achievement such that the validity of self-assessments has been called into question. Based on the reanalysis of two previously published studies and an analysis of two original studies, we show that grouped self-assessments may be good predictors of and hence valid measures of performance at the group level, an outcome commonly used in program evaluation studies. We found statistically significant correlation coefficients (between 0.56 and 0.87), when comparing across performance items using the group means of self-assessments with the group means of individual achievement on criterion tests. We call for further research into the conditions and circumstances in which grouped self-assessments are used, so that they can be employed more effectively and confidently by program evaluators, decision makers, and researchers.
引用
收藏
页码:457 / 469
页数:13
相关论文
共 26 条
[1]   Shoestring evaluation: Designing impact evaluations under budget, time and data constraints [J].
Bamberger, M ;
Rugh, J ;
Church, M ;
Fort, L .
AMERICAN JOURNAL OF EVALUATION, 2004, 25 (01) :5-37
[2]   Medical students' self-assessment of performance: Results from three meta-analyses [J].
Blanch-Hartigan, Danielle .
PATIENT EDUCATION AND COUNSELING, 2011, 84 (01) :3-9
[3]   Using self-assessments to detect workshop success - Do they work? [J].
D'Eon, Marcel ;
Sadownik, Leslie ;
Harrison, Alexandra ;
Nation, Jill .
AMERICAN JOURNAL OF EVALUATION, 2008, 29 (01) :92-98
[4]   Evaluation of a teaching workshop for residents at the University of Saskatchewan: A pilot study [J].
D'Eon, MF .
ACADEMIC MEDICINE, 2004, 79 (08) :791-797
[5]   Accuracy of physician self-assessment compared with observed measures of competence - A systematic review [J].
Davis, David A. ;
Mazmanian, Paul E. ;
Fordis, Michael ;
Van Harrison, R. ;
Thorpe, Kevin E. ;
Perrier, Laure .
JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION, 2006, 296 (09) :1094-1102
[6]   Supervisor and Self-Ratings of Graduates From a Medical School With a Problem-Based Learning and Standard Curriculum Track [J].
Distlehorst, Linda H. ;
Dawson, Beth K. ;
Klamen, Debra L. .
TEACHING AND LEARNING IN MEDICINE, 2009, 21 (04) :291-298
[7]   How can I know what I don't know? Poor self assessment in a well-defined domain [J].
Eva, KW ;
Cunnington, JPW ;
Reiter, HI ;
Keane, DR ;
Norman, GR .
ADVANCES IN HEALTH SCIENCES EDUCATION, 2004, 9 (03) :211-224
[8]  
Gall M. D., 2003, Educational Research: An Introduction, V7th ed.
[9]   RESPONSE-SHIFT BIAS - A PROBLEM IN EVALUATING INTERVENTIONS WITH PRE-POST SELF-REPORTS [J].
HOWARD, GS .
EVALUATION REVIEW, 1980, 4 (01) :93-106
[10]  
Kirkpatrick D.L., 2006, Evaluating training programs: The four levels, V3rd