Consistency in the ratings of behaviors of communicatively impaired autistic and non-autistic preschool children

Typically, children with disabilities are evaluated clinically by a number of professionals with different backgrounds whose task is to provide a diagnosis and an intervention plan. This study was carried out to describe interrater agreement for pairs of independent observers rating with different instruments the behaviors of 505 communicatively impaired autistic and nonautistic preschool children. Observers were parents, teachers, neurologists, and psychiatrists. Parents and teachers responded to behavioral questionnaires, neurologists filled out the mental status part of a standardized neurologic evaluation, and psychiatrists an observational questionnaire. All four types of observers rated sociability, language, play, attention, stereotyped, and other aberrant behaviors. Agreement between pairs of raters was significant but moderate. Owing to range restriction with smaller numbers of subjects, agreement decreased for ratings of subsamples divided according to diagnosis, cognitive level, or age. There were some differences among observers' ratings of the severity of particular categories of behaviors, with physicians generally viewing the children as more severely impaired and teachers as least impaired. Interrater agreement was not enhanced when parents and teachers rated similarly worded behavioral items. Modest interrater agreement in this study, like agreement among disparate raters of children's behaviors in other studies, suggests that observers are sensitive to different aspects of behavior and that their ratings are more likely to be complementary than unreliable.