Inconsistencies of recently proposed citation impact indicators and how to avoid them

被引:13
作者
Schreiber, Michael [1 ]
机构
[1] Tech Univ Chemnitz, Inst Phys, D-09107 Chemnitz, Germany
来源
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY | 2012年 / 63卷 / 10期
关键词
quantitative research; physical sciences; INDEX;
D O I
10.1002/asi.22703
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
It is shown that under certain circumstances in particular for small data sets, the recently proposed citation impact indicators I3(6PR) and R(6,k) behave inconsistently when additional papers or citations are taken into consideration. Three simple examples are presented, in which the indicators fluctuate strongly and the ranking of scientists in the evaluated group is sometimes completely mixed up by minor changes in the database. The erratic behavior is traced to the specific way in which weights are attributed to the six percentile rank classes, specifically for the tied papers. For 100 percentile rank classes, the effects will be less serious. For the six classes, it is demonstrated that a different way of assigning weights avoids these problems, although the nonlinearity of the weights for the different percentile rank classes can still lead to (much less frequent) changes in the ranking. This behavior is not undesired because it can be used to correct for differences in citation behavior in different fields. Remaining deviations from the theoretical value R(6,k)?=?1.91 can be avoided by a new scoring rule: the fractional scoring. Previously proposed consistency criteria are amended by another property of strict independence at which a performance indicator should aim.
引用
收藏
页码:2062 / 2073
页数:12
相关论文
共 11 条
[1]  
[Anonymous], 2010, SCI ENG IND
[2]   Further steps towards an ideal method of measuring citation performance: The avoidance of citation (ratio) averages in field-normalization [J].
Bornmann, Lutz ;
Mutz, Ruediger .
JOURNAL OF INFORMETRICS, 2011, 5 (01) :228-230
[3]   Percentile ranks and the integrated impact indicator (I3) [J].
Leydesdorff, Loet ;
Bornmann, Lutz .
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY, 2012, 63 (09) :1901-1902
[4]   Integrated Impact Indicators Compared With Impact Factors: An Alternative Research Design With Policy Implications [J].
Leydesdorff, Loet ;
Bornmann, Lutz .
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY, 2011, 62 (11) :2133-2146
[5]   Turning the Tables on Citation Analysis One More Time: Principles for Comparing Sets of Documents [J].
Leydesdorff, Loet ;
Bornmann, Lutz ;
Mutz, Ruediger ;
Opthof, Tobias .
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY, 2011, 62 (07) :1370-1381
[6]   Percentile Rank and Author Superiority Indexes for Evaluating Individual Journal Articles and the Author's Overall Citation Performance [J].
Pudovkin, A. I. ;
Garfield, E. .
COLLNET JOURNAL OF SCIENTOMETRICS AND INFORMATION MANAGEMENT, 2009, 3 (02) :3-10
[7]  
Rousseau R, 2011, PERCENTILE RANK SCOR
[8]   Basic Properties of Both Percentile Rank Scores and the I3 Indicator [J].
Rousseau, Ronald .
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY, 2012, 63 (02) :416-420
[9]   An empirical investigation of the g-index for 26 physicists in comparison with the h-Index, the A-index, and the R-index [J].
Schreiber, Michael .
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY, 2008, 59 (09) :1513-1522
[10]   A Case Study of the Modified Hirsch Index hm Accounting for Multiple Coauthors [J].
Schreiber, Michael .
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY, 2009, 60 (06) :1274-1282