Scoring Subjectivity and Item Performance on Measures Used to Assess Violence Risk: The PCL-R and HCR-20 as Exemplars

Although reliability is essential to validity, most research on violence risk assessment tools has paid little attention to strategies for improving rater agreement. The authors evaluated the degree to which perceived subjectivity in scoring guidelines for items from two measures-the Psychopathy Checklist-Revised (PCL-R) and the Historical, Clinical, Risk Management-20 (HCR-20)-were related to indices of interrater agreement reported in the research literature. Nine doctoral students with experience scoring the PCL-R and HCR-20 showed a high level of agreement with respect to the amount of subjectivity required to score items. Decreased subjectivity was associated with increased rater agreement for both tools. For the PCL-R, items on aggregate measures that have the strongest effects in predictive validity research (i.e., Factor 2, Facet 4) were rated as the least subjective. For the HCR-20, items on the Historical scale were rated as the least subjective. Implications for measure development and refinement are discussed.