Risk assessment for people with mental health problems: a pilot study of reliability in working practice

INTRODUCTION: This paper describes a pilot study of reliability in the risk assessment of people with mental health problems. Specifically, we explore the evidence for professional and gender bias in ratings, in addition to the general level of agreement between raters. METHOD: Six professional groups (psychiatrists, junior psychiatric doctors, nurses, community psychiatric nurses, social workers and occupational therapists) participated in the study and rated 159 patients on a nine-item scale which assessed different components of risk. RESULTS: Contrary to some earlier work, we found no clear evidence that any one group consistently rated more extremely than any other group. Women were more cautious than men in their ratings, and this concurs with previous studies. Finally, a reliability study of randomly selected pairs of raters showed only moderate levels of agreement and, in some instances, the levels of disagreement were high enough to warrant concern. CONCLUSION: These findings are discussed in the context of current risk assessment practice and the problems associated with investigating reliability in naturalistic settings and designing appropriate rating tools for risk.