National Internal Medicine Milestone Ratings: Validity Evidence From Longitudinal Three-Year Follow-up

被引：24

作者：

Hauer, Karen E. ^{[1
]}

Vandergrift, Jonathan ^{[2
]}

Lipner, Rebecca S. ^{[3
]}

Holmboe, Eric S. ^{[4
]}

Hood, Sarah ^{[5
]}

McDonald, Furman S. ^{[6
]}

机构：

[1] Univ Calif San Francisco, Dept Med, San Francisco, CA USA

[2] ABIM, Philadelphia, PA USA

[3] ABIM, Assessment & Res, Philadelphia, PA USA

[4] Accreditat Council Grad Med Educ, Milestones Dev & Evaluat, Chicago, IL USA

[5] ABIM, Initial Certificat, Philadelphia, PA USA

[6] ABIM, Acad & Med Affairs, Philadelphia, PA USA

来源：

ACADEMIC MEDICINE | 2018年 / 93卷 / 08期

关键词：

CERTIFICATION EXAMINATION SCORES; REPORTING MILESTONES; COMPETENCE; EDUCATION; PERFORMANCE; RESIDENTS; ASSOCIATION;

D O I：

10.1097/ACM.0000000000002234

中图分类号：

G40 [教育学];

学科分类号：

040101 ; 120403 ;

摘要：

Purpose To evaluate validity evidence for internal medicine milestone ratings across programs for three resident cohorts by quantifying not assessable ratings; reporting mean longitudinal milestone ratings for individual residents; and correlating medical knowledge ratings across training years with certification examination scores to determine predictive validity of milestone ratings for certification outcomes. Method This retrospective study examined milestone ratings for postgraduate year (PGY) 1-3 residents in U.S. internal medicine residency programs. Data sources included milestone ratings, program characteristics, and certification examination scores. Results Among 35,217 participants, there was a decreased percentage with not assessable ratings across years: 1,566 (22.5%) PGY1s in 2013-2014 versus 1,219 (16.6%) in 2015-2016 (P = .01), and 342 (5.1%) PGY3s in 2013-2014 versus 177 (2.6%) in 2015-2016 (P = .04). For individual residents with three years of ratings, mean milestone ratings increased from around 3 (behaviors of an early learner or advancing resident) in PGY1 (ranging from a mean of 2.73 to 3.19 across subcompetencies) to around 4 (ready for unsupervised practice) in PGY3 (mean of 4.00 to 4.22 across subcompetencies, P < .001 for all subcompetencies). For each increase of 0.5 units in two medical knowledge (MK1, MK2) subcompetency ratings, the difference in examination scores for PGY3s was 19.5 points for MK1 (P < .001) and 19.0 for MK2 (P < .001). Conclusions These findings provide evidence of validity of the milestones by showing how training programs have applied them over time and how milestones predict other training outcomes.

引用

页码：1189 / 1204

页数：16

共 36 条

[1] Initial Validity Analysis of the Emergency Medicine Milestones [J].

Beeson, Michael S. ;

Holmboe, Eric S. ;

Korte, Robert C. ;

Nasca, Thomas J. ;

Brigham, Timothy ;

Russ, Chad M. ;

Whitley, Cameron T. ;

Reisdorff, Earl J. .

ACADEMIC EMERGENCY MEDICINE, 2015, 22 (07) :838-844

[2] CONTROLLING THE FALSE DISCOVERY RATE - A PRACTICAL AND POWERFUL APPROACH TO MULTIPLE TESTING [J].

BENJAMINI, Y ;

HOCHBERG, Y .

JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-STATISTICAL METHODOLOGY, 1995, 57 (01) :289-300

[3] Advancing Competency-Based Medical Education: A Charter for Clinician-Educators [J].

Carraccio, Carol ;

Englander, Robert ;

Van Melle, Elaine ;

ten Cate, Olle ;

Lockyer, Jocelyn ;

Chan, Ming-Ka ;

Frank, Jason R. ;

Snell, Linda S. .

ACADEMIC MEDICINE, 2016, 91 (05) :645-649

[4] The Internal Medicine Reporting Milestones and the Next Accreditation System [J].

Caverzagie, Kelly J. ;

Iobst, William F. ;

Aagaard, Eva M. ;

Hood, Sarah ;

Chick, Davoren A. ;

Kane, Gregory C. ;

Brigham, Timothy P. ;

Swing, Susan R. ;

Meade, Lauren B. ;

Bazari, Hasan ;

Bush, Roger W. ;

Kirk, Lynne M. ;

Green, Michael L. ;

Hinchey, Kevin T. ;

Smith, Cynthia D. .

ANNALS OF INTERNAL MEDICINE, 2013, 158 (07) :557-+

[5]

Cook David A, 2006, Am J Med, V119

[6] Toward a definition of competency-based education in medicine: a systematic review of published definitions [J].

Frank, Jason R. ;

Mungroo, Rani ;

Ahmad, Yasmine ;

Wang, Mimi ;

De Rossi, Stefanie ;

Horsley, Tanya .

MEDICAL TEACHER, 2010, 32 (08) :631-637

[7]

Green Michael L, 2009, J Grad Med Educ, V1, P5, DOI 10.4300/01.01.0003

[8] Correlations Between Ratings on the Resident Annual Evaluation Summary and the Internal Medicine Milestones and Association With ABIM Certification Examination Scores Among US Internal Medicine Residents, 2013-2014 [J].

Hauer, Karen E. ;

Vandergrift, Jonathan ;

Hess, Brian ;

Lipner, Rebecca S. ;

Holmboe, Eric S. ;

Hood, Sarah ;

Iobst, William ;

Hamstra, Stanley J. ;

McDonald, Furman S. .

JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION, 2016, 316 (21) :2253-2262

[9] The Internal Medicine Reporting Milestones: Cross-sectional Description of Initial Implementation in US Residency Programs [J].

Hauer, Karen E. ;

Clauser, Jerome ;

Lipner, Rebecca S. ;

Holmboe, Eric S. ;

Caverzagie, Kelly ;

Hamstra, Stanley J. ;

Hood, Sarah ;

Iobst, William ;

Warm, Eric ;

McDonald, Furman S. .

ANNALS OF INTERNAL MEDICINE, 2016, 165 (05) :356-+

[10] Reviewing Residents' Competence: A Qualitative Study of the Role of Clinical Competency Committees in Performance Assessment [J].

Hauer, Karen E. ;

Chesluk, Benjamin ;

Iobst, William ;

Holmboe, Eric ;

Baron, Robert B. ;

Boscardin, Christy K. ;

ten Cate, Olle ;

O'Sullivan, Patricia S. .

ACADEMIC MEDICINE, 2015, 90 (08) :1084-1092

← 1 2 3 4 →