Intercoder reliability indices: disuse, misuse, and abuse

被引：0
作者：
Guangchao Charles Feng
机构：
[1] Jinan University,School of Journalism and Communication
来源：
Quality & Quantity | 2014年 / 48卷
关键词：
Intercoder reliability; Content analysis; Misuse;
D O I：
暂无
中图分类号：
学科分类号：
摘要：
Although intercoder reliability has been considered crucial to the validity of a content study, the choice among them has been controversial. This study analyzed all the content studies published in the two major communication journals that reported intercoder reliability, aiming to find how scholars conduct intercoder reliability test. The results revealed that some intercoder reliability indices were misused persistently concerning the levels of measurement, the number of coders, and the means of reporting reliability over the past 30 years. Implications of misuse, disuse, and abuse were discussed, and suggestions regarding proper choice of indices in various situations were made at last.
引用
收藏
页码：1803 / 1815
页数：12
相关论文
共 35 条
[1]  
Artstein R(2008)Inter-coder agreement for computational linguistics Comput. Linguist. 34 555-596
[2]  
Poesio M(1981)Coefficient kappa: some uses, misuses, and alternatives Educ. Psychol. Meas. 41 687-429
[3]  
Brennan R(1993)Bias, prevalence and kappa J. Clin. Epidemiology 46 423-558
[4]  
Prediger D(1990)High agreement but low kappa: II. Resolving the paradoxes* 1 J. Clin. Epidemiol. 43 551-46
[5]  
Byrt T.(1960)A coefficient of agreement for nominal scales Educ. Psychol. Meas. 20 37-334
[6]  
Bishop J.(1980)Integration and generalization of kappas for multiple raters Psychol. Bull. 88 322-549
[7]  
Carlin J.B.(1951)Coefficient alpha and the internal structure of tests Psychometrika 16 297-76
[8]  
Cicchetti D(1990)High agreement but low kappa: I. The problems of two paradoxes J. Clin. Epidemiol. 43 543-382
[9]  
Feinstein A(1970)A note on estimating the reliability of categorical data Educ. Psychol. Meas. 30 71-9
[10]  
Cohen J(1971)Measuring nominal scale agreement among many raters Psychol. Bull. 76 378-48
1234