Improvement of inter-rater reliability of PANSS items and subscales by a standardized rater training

The present evaluation focused on the inter-rater reliability of single items and subscales of the Positive and Negative Syndrome Scale (PANSS) before and after a standardized rater training. The results of two independent studies comprising n=23 (study I) and n=12 (study II) psychiatrists and videotaped interviews with schizophrenic patients were analysed. Chance-corrected coefficients of rating agreement with expert standards (weighted kappa) were computed for single items and subscales of the PANSS. The results dearly demonstrate the importance of rater trainings. After three training sessions, 90% of the PANSS items reached an acceptable level of reliability (kappa(W)>0.40) in both evaluated studies, and for 80% of the PANSS items values of kappa(W)>0.60 were achieved.