Generalizability of Universal Screening Measures for Behavioral and Emotional Risk

Data derived from universal screening procedures are increasingly utilized by schools to identify and provide additional support to students at risk for behavioral and emotional concerns. As screening has the potential to be resource intensive, effort has been placed on the development of efficient screening procedures, including brief behavior rating scales (BBRS). This study utilized classical test theory and generalizability theory to examine the extent to which differences among students, raters, occasions, and screening measures affect the amount of variance in data derived from universal screening procedures. Teacher pairs from three middle school classrooms completed two BBRS during fall and spring for each student in their classrooms. Correlation coefficients examining interrater reliability, test-retest reliability, and convergent validity were generally strong. Generalizability analyses indicated that the majority of variance in teacher ratings was attributable to student differences across all score comparisons, but differences between teacher ratings for particular students accounted for relatively large percentages of error variance among student behavior ratings. Although decision studies suggested that increasing the number of screening occasions resulted in more generalizable data, they also demonstrated that increasing the number of raters not only resulted in more generalizable data but also procedures that are more efficient.