Agreement and kappa-type indices

被引:53
作者
De Mast, Jeroen [1 ]
机构
[1] Univ Amsterdam, NL-1018 TV Amsterdam, Netherlands
关键词
categorical data; gauge capability analysis; measurement system analysis; nominal data; reliability; reproducibility;
D O I
10.1198/000313007X192392
中图分类号
O21 [概率论与数理统计]; C8 [统计学];
学科分类号
020208 ; 070103 ; 0714 ;
摘要
Kappa-type indices use the concept of agreement to express the reproducibility of nominal measurements. This article grounds kappa-type indices in statistical modeling, making explicit the underlying premises and assumptions. We critically review whether the interpretation of the kappa index as a chance-corrected probability of agreement can be substantiated. Further, we show that the so-called paradoxical behavior of the kappa index is explained from the fact that it is a measure of predictive association, rather than a pure measure of reproducibility. We discuss a number of alternative forms, critically examining whether they can be translated in tangible real-life interpretations.
引用
收藏
页码:148 / 153
页数:6
相关论文
共 24 条
[1]  
Allen M.J., 1979, Introduction to measurement theory
[2]  
BENNETT EM, 1954, PUBLIC OPIN QUART, V18, P303
[3]   COEFFICIENT KAPPA - SOME USES, MISUSES, AND ALTERNATIVES [J].
BRENNAN, RL ;
PREDIGER, DJ .
EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 1981, 41 (03) :687-699
[4]   A review of methods for measurement systems capability analysis [J].
Burdick, RK ;
Borror, CM ;
Montgomery, DC .
JOURNAL OF QUALITY TECHNOLOGY, 2003, 35 (04) :342-354
[5]   A COEFFICIENT OF AGREEMENT FOR NOMINAL SCALES [J].
COHEN, J .
EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 1960, 20 (01) :37-46
[6]   INTEGRATION AND GENERALIZATION OF KAPPAS FOR MULTIPLE RATERS [J].
CONGER, AJ .
PSYCHOLOGICAL BULLETIN, 1980, 88 (02) :322-328
[7]   MEASURING AGREEMENT FOR MULTINOMIAL DATA [J].
DAVIES, M ;
FLEISS, JL .
BIOMETRICS, 1982, 38 (04) :1047-1051
[8]  
DEMAST J, IN PRESS J QUALITY T
[9]   HIGH AGREEMENT BUT LOW KAPPA .1. THE PROBLEMS OF 2 PARADOXES [J].
FEINSTEIN, AR ;
CICCHETTI, DV .
JOURNAL OF CLINICAL EPIDEMIOLOGY, 1990, 43 (06) :543-549
[10]  
FEINSTEIN AR, 1990, J CLIN EPIDEMIOL, V43, P553