A meta-evaluation of scientific research proposals: Different ways of comparing rejected to awarded applications

被引:59
作者
Bornmann, Lutz [1 ]
Leydesdorff, Loet [2 ]
Van den Besselaar, Peter [3 ,4 ]
机构
[1] ETH, CH-8092 Zurich, Switzerland
[2] Univ Amsterdam, ASCoR, NL-1012 CX Amsterdam, Netherlands
[3] Rathenau Inst, Sci Syst Assessment Dept, NL-2593 HW The Hague, Netherlands
[4] Vrije Univ Amsterdam, Dept Org Sci, Amsterdam, Netherlands
关键词
Grant allocation; Peer review; Bibliometric quality indicators; Convergent validity and predictive validity; Error; Citation rate; h-Index; H-INDEX; PERFORMANCE; INDICATORS; SCIENCE;
D O I
10.1016/j.joi.2009.10.004
中图分类号
TP39 [计算机的应用];
学科分类号
081203 ; 0835 ;
摘要
Combining different data sets with information on grant and fellowship applications submitted to two renowned funding agencies, we are able to compare their funding decisions (award and rejection) with scientometric performance indicators across two fields of science (life sciences and social sciences). The data sets involve 671 applications in social sciences and 668 applications in life sciences. In both fields, awarded applicants perform on average better than all rejected applicants. If only the most preeminent rejected applicants are considered in both fields, they score better than the awardees on citation impact. With regard to productivity we find differences between the fields. While the awardees in life sciences outperform on average the most preeminent rejected applicants, the situation is reversed in social sciences. (C) 2009 Elsevier Ltd. All rights reserved.
引用
收藏
页码:211 / 220
页数:10
相关论文
共 33 条
[1]   INEQUALITY AND SCIENTIFIC PRODUCTIVITY [J].
ALLISON, PD .
SOCIAL STUDIES OF SCIENCE, 1980, 10 (02) :163-179
[2]  
[Anonymous], 1976, EVALUATIVE BIBLIOMET
[3]  
BORNMANN L, 2007, ANN REV INF IN PRESS
[4]   Is the h index related to (standard) bibliometric measures and to the assessments by peers?: An investigation of the h index by using molecular life sciences data [J].
Bornmann, Lutz ;
Wallon, Gerlind ;
Ledin, Anna .
RESEARCH EVALUATION, 2008, 17 (02) :149-156
[5]   Convergent validation of peer review decisions using the h index -: Extent of and reasons for type I and type II errors [J].
Bornmann, Lutz ;
Daniel, Hans-Dieter .
JOURNAL OF INFORMETRICS, 2007, 1 (03) :204-213
[6]   Are there better indices for evaluation purposes than the h index?: a comparison of nine different variants of the h index using data from biomedicine [J].
Bornmann, Lutz ;
Mutz, Ruediger ;
Daniel, Hans-Dieter .
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY, 2008, 59 (05) :830-837
[7]   Multiple publication on a single research study: Does it pay? The influence of number of research articles on total citation counts in biomedicine [J].
Bornmann, Lutz ;
Daniel, Hans-Dieter .
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY, 2007, 58 (08) :1100-1107
[8]   Does the Committee Peer Review Select the Best Applicants for Funding? An Investigation of the Selection Process for Two European Molecular Biology Organization Programmes [J].
Bornmann, Lutz ;
Wallon, Gerlind ;
Ledin, Anna .
PLOS ONE, 2008, 3 (10)
[9]  
DEHAAN J, 1994, RES GROUPS DUTCH SOC
[10]  
Elkana Y., 1978, Toward a metric of science: The advent of science indicators