Effect of attendance at a training session on peer reviewer quality and performance

被引:57
作者
Callaham, ML [1 ]
Wears, RL [1 ]
Waeckerle, JF [1 ]
机构
[1] Univ Calif San Francisco, Div Emergency Med, Dept Med, San Francisco, CA 94143 USA
关键词
D O I
10.1016/S0196-0644(98)70007-1
中图分类号
R4 [临床医学];
学科分类号
1002 ; 100602 ;
摘要
Study objective: To determine whether attendance at a voluntary training workshop improves quality ratings of medical journal peer reviewers. Methods: Peer reviewers for Annals of Emergency Medicine who completed two or more reviews during the 20 months before or the 20 months after October 1995 were eligible. Reviews were routinely rated by editors on a subjective 5-point quality scale. Comparisons were made between reviewers who chose to attend a 4-hour workshop on peer review sponsored by the journal in 1995 (attendees) and 2 groups of reviewers who did not attend. controls matched for review quality and number of reviews completed before the workshop, and unmatched controls. Guest reviewers were excluded. Results: A total of 298 reviewers completed 1906 reviews before the workshop and 2,194 after the workshop; 2,117 of these reviews were rated by editors. Forty-five attendees participated in the workshop, 39 of whom had sufficient ratings for analysis. Matched controls were almost identical in performance to attendees, but unmatched controls had performed fewer reviews and had lower average ratings before the workshop. There was no significant change in any performance measurement after the workshop, including average quality rating, percent change in quality rating, odds ratio for recommending acceptance, and odds ratio for congruence with editor's decision. Conclusion: In a self-selected group of experienced reviewers who attended a 4-hour workshop on peer review, no effect could be identified in subsequent performance as measured by editors' quality ratings or reviewer performance statistics.
引用
收藏
页码:318 / 322
页数:5
相关论文
共 13 条
[1]   Who reviews the reviewers? Feasibility of using a fictitious manuscript to evaluate peer reviewer performance [J].
Baxt, WG ;
Waeckerle, JF ;
Berlin, JA ;
Callaham, ML .
ANNALS OF EMERGENCY MEDICINE, 1998, 32 (03) :310-317
[2]  
BAXT WG, 1996, ACAD EMERG MED, V3, P504
[3]  
BAZARIAN J, 1997, ACAD EMERG MED, V4, P430
[4]  
CALLAHAM M, 1997, CBE VIEWS COUNCIL BI, V20, P192
[5]   Reliability of editors' subjective quality ratings of peer reviews of manuscripts [J].
Callaham, ML ;
Baxt, WG ;
Waeckerie, JF ;
Wears, RL .
JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION, 1998, 280 (03) :229-231
[6]   THE RELIABILITY OF PEER-REVIEW FOR MANUSCRIPT AND GRANT SUBMISSIONS - A CROSS-DISCIPLINARY INVESTIGATION [J].
CICCHETTI, DV .
BEHAVIORAL AND BRAIN SCIENCES, 1991, 14 (01) :119-134
[7]  
Langkamp D L, 1992, Fam Med, V24, P528
[8]   IMPACT OF A MEDICAL JOURNAL CLUB ON HOUSE-STAFF READING HABITS, KNOWLEDGE, AND CRITICAL-APPRAISAL SKILLS - A RANDOMIZED CONTROL TRIAL [J].
LINZER, M ;
BROWN, JT ;
FRAZIER, LM ;
DELONG, ER ;
SIEGEL, WC .
JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION, 1988, 260 (17) :2537-2541
[9]   ETHICS, ECONOMICS, AND THE PUBLICATION POLICIES OF MAJOR MEDICAL JOURNALS [J].
SCHULMAN, K ;
SULMASY, DP ;
RONEY, D .
JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION, 1994, 272 (02) :154-156
[10]  
STEINMANN WC, UNPUB SURVEY J EDITO