Inter-rater reliability of the Early Assessment Risk List Version 3 (EARL-V3) for children displaying antisocial behavior

Background: Identification of risks is foundational to the development of strategies to prevent conduct problems in children. The Early Assessment Risk List Version 3 (EARL-V3) is a Structured Professional Judgment (SPJ) instrument designed for application to children exhibiting antisocial behavior. The revised EARL-V3 (2021) encourages users to be responsive to gender and cultural issues and is designed to assist in treatment planning. While previous versions of the EARL have undergone extensive testing, reliability has not been established for the new version. Methods: The present study yields Intraclass Correlation Coefficients (ICCs) for the three EARL-V3 subscales and the total score. Additional analyses are based on Gwet's AC1 method for examining individual items. Four rater pairs assessed a selection of 124 children (50 girls, 74 boys; six to 12 years). Case files included child and caregiver interviews and clinical assessments. Results: We found moderate to excellent agreement for two of the three subscales and the total score, and poor to moderate agreement for one subscale. The individual items ranged from fair to almost perfect agreement. The Overall Clinical Risk Judgement revealed fair to moderate inter-rater agreement. The results largely remained when analyzed based on gender. Some differences were seen for individual risk factors. Conclusion: The EARL-V3 has a good level of inter-rater reliability, comparable or superior to previous versions of the EARL and to other SPJ instruments.