The predictive validity of peer review: A selective review of the judgmental forecasting qualities of peers, and implications for innovation in science

被引:28
作者
Benda, Wim G. G. [1 ]
Engels, Tim C. E. [1 ,2 ]
机构
[1] Univ Antwerp, Ctr R&D Monitoring, B-2020 Antwerp, Belgium
[2] Antwerp Maritime Acad, Dept Social Sci & Languages, B-2030 Antwerp, Belgium
关键词
Advice taking; Cognitive bias; Decision-making; Expert advice; Group decision making; Reliability; CUM LAUDE DOCTORATES; DECISION-MAKING; BIBLIOMETRIC INDICATORS; GRANT APPLICATIONS; CITATION ANALYSIS; JOURNALS; RELIABILITY; PUBLICATION; PERFORMANCE; BIAS;
D O I
10.1016/j.ijforecast.2010.03.003
中图分类号
F [经济];
学科分类号
02 ;
摘要
In this review we investigate what the available data on the predictive validity of peer review can add to our understanding of judgmental forecasting. We found that peer review attests to the relative success of judgmental forecasting by experts. Both manuscript and group-based peer review allow, on average, for accurate decisions to be made. However, tension exists between peer review and innovative ideas, even though the latter underlie scientific advance. This points to the danger of biases and preconceptions in judgments. We therefore formulate two proposals for enhancing the likelihood of innovative work. (C) 2010 International Institute of Forecasters. Published by Elsevier B.V. All rights reserved.
引用
收藏
页码:166 / 182
页数:17
相关论文
共 103 条
[11]   DOES THE NEED FOR AGREEMENT AMONG REVIEWERS INHIBIT THE PUBLICATION OF CONTROVERSIAL FINDINGS [J].
ARMSTRONG, JS ;
HUBBARD, R .
BEHAVIORAL AND BRAIN SCIENCES, 1991, 14 (01) :136-136
[12]  
BAKANIC V, 1990, AM SOCIOL, V21, P373
[13]   Peer Review and the Social Construction of Knowledge in the Management Discipline [J].
Bedeian, Arthur G. .
ACADEMY OF MANAGEMENT LEARNING & EDUCATION, 2004, 3 (02) :198-216
[14]   Advice taking and decision-making: An integrative literature review, and implications for the organizational sciences [J].
Bonaccio, Silvia ;
Dalal, Reeshad S. .
ORGANIZATIONAL BEHAVIOR AND HUMAN DECISION PROCESSES, 2006, 101 (02) :127-151
[15]   Selection of research fellowship recipients by committee peer review. Reliability, fairness and predictive validity of Board of Trustees' decisions [J].
Bornmann, L ;
Daniel, HD .
SCIENTOMETRICS, 2005, 63 (02) :297-320
[16]   What do citation counts measure? A review of studies on citing behavior [J].
Bornmann, Luti ;
Daniel, Hans-Dieter .
JOURNAL OF DOCUMENTATION, 2008, 64 (01) :45-80
[17]   Selecting manuscripts for a high-impact journal through peer review:: A citation analysis of communications that were accepted by Angewandte Chemie International Edition, or rejected but published elsewhere [J].
Bornmann, Lutz ;
Daniel, Hans-Dieter .
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY, 2008, 59 (11) :1841-1852
[19]   The manuscript reviewing process: Empirical research on review requests, review sequences, and decision rules in peer review [J].
Bornmann, Lutz ;
Daniel, Hans-Dieter .
LIBRARY & INFORMATION SCIENCE RESEARCH, 2010, 32 (01) :5-12
[20]   Extent of type I and type II errors in editorial decisions: A case study on Angewandte Chemie International Edition [J].
Bornmann, Lutz ;
Daniel, Hans-Dieter .
JOURNAL OF INFORMETRICS, 2009, 3 (04) :348-352