FACTORS AFFECTING THE RELIABILITY OF RATINGS OF STUDENTS CLINICAL SKILLS IN A MEDICINE CLERKSHIP

被引：46
作者：
CARLINE, JD
PAAUW, DS
THIEDE, KW
RAMSEY, PG
机构：
[1] Department of Medical Education, SC-45, University of Washington School of Medicine, Seattle, 98195, WA
[2] the Department of Medicine, University of Washington School of Medicine, Seattle, Washington
来源：
JOURNAL OF GENERAL INTERNAL MEDICINE | 1992年 / 7卷 / 05期
关键词：
MEDICAL STUDENTS; CLINICAL CLERKSHIP; RELIABILITY; EDUCATIONAL MEASUREMENT;
D O I：
10.1007/BF02599454
中图分类号：
R19 [保健组织与事业（卫生事业管理）];
学科分类号：
摘要：
Objective: To determine the overall reliability and factors that might affect the reliability of ratings of students' clinical skills in a medicine clerkship. Design: A nine-item instrument was used to evaluate students' clinical skills. Raters were also asked to provide a grade of each student's overall clinical performance. Generalizability studies were performed to estimate the reliability of the ratings. The effects of rater experience and clerkship setting were investigated by regression analysis. Setting: Teaching hospitals and community-based sites in three Northwestern states. Participants: All students (328) who bad completed the 12-week clerkship in internal medicine at one medical school during the academic wars 1987-1989. Raters included attending physicians, chief residents, and other residents. Results: Seven observations were needed to provide a reliable rating of the overall clinical grade. More observations were needed to obtain reliable ratings for individual items, ranging from seven observations needed for the rating of data gathering skills to 27 observations needed for the rating of interpersonal relationships with patients Rater experience and clerkship setting (i.e., teaching hospitals vs. community based clinics) were found, in general, not to affect significantly the ratings received by students. Conclusions: Reliable ratings of students' overall clinical skills, including overall clinical grades, can be achieved by collecting a minimum of seven observations. More observations are needed to measure reliably the interpersonal aspects of clinical performance. These findings support the use of performance ratings to evaluate clinical skills and knowledge of students in clerkship settings.
引用
收藏
页码：506 / 510
页数：5
相关论文
共 11 条
  • [1] Magarian G.J., Mazur D.J., Evaluation of students in medicine clerkships, Acad. Med, 65, pp. 341-5, (1990)
  • [2] Tonesk X., Buchanan R.G., An AAMC pilot study by 10 medical schools of clinical evaluation of students, J Med Educ, 62, pp. 707-18, (1987)
  • [3] Hunt D.D., Carline J.D., Tonesk X., Et al., Types of problem students encountered by clinical teachers on clerkships, J Med Educ, 23, pp. 14-8, (1989)
  • [4] Harden R.M., Gleeson F.A., Assessment of clinical competence using an Objective Structured Clinical Examination (OSCE), Med Educ, 13, pp. 41-54, (1979)
  • [5] Maxim B.R., Dielman T.E., Dimensionality, internal consistency and interrater reliability of clinical performance ratings, Med Educ, 21, pp. 130-7, (1987)
  • [6] Hull A.L., Medical student performance: a comparison of house officer and attending staff as evaluators, Eval Health Prof, 5, pp. 87-94, (1982)
  • [7] Carline J.D., Cook C.E., Lennard E.S., Coluccio G.M., Norman N.L., Resident and faculty differences in student evaluations: implications for changes in a clerkship evaluation system, Surgery, 100, pp. 88-94, (1986)
  • [8] Raey P.G., Shannon N.F., Fleming L., Et al., Use of objective examinations in medicine clerkships, Am J Med, 81, pp. 669-74, (1986)
  • [9] Shavelson R.J., Webb N.M., Generalizability theory: a primer, (1991)
  • [10] Winer B.J., Statistics: principles in experimental design, (1971)
12