Consistency in the ratings of behaviors of communicatively impaired autistic and non-autistic preschool children

Abstract

Typically, children with disabilities are evaluated clinically by a number of professionals with different backgrounds whose task is to provide a diagnosis and an intervention plan. This study was carried out to describe interrater agreement for pairs of independent observers rating with different instruments the behaviors of 505 communicatively impaired autistic and nonautistic preschool children. Observers were parents, teachers, neurologists, and psychiatrists. Parents and teachers responded to behavioral questionnaires, neurologists filled out the mental status part of a standardized neurologic evaluation, and psychiatrists an observational questionnaire. All four types of observers rated sociability, language, play, attention, stereotyped, and other aberrant behaviors. Agreement between pairs of raters was significant but moderate. Owing to range restriction with smaller numbers of subjects, agreement decreased for ratings of subsamples divided according to diagnosis, cognitive level, or age. There were some differences among observers' ratings of the severity of particular categories of behaviors, with physicians generally viewing the children as more severely impaired and teachers as least impaired. Interrater agreement was not enhanced when parents and teachers rated similarly worded behavioral items. Modest interrater agreement in this study, like agreement among disparate raters of children's behaviors in other studies, suggests that observers are sensitive to different aspects of behavior and that their ratings are more likely to be complementary than unreliable.

Parent and Teacher Concordance on the Social Responsiveness Scale for Children with Autism

“…Our results are consistent with prior investigations which have shown that agreement is usually higher when the disorder is more severely impairing (Pearson et al, 2012; Rapin et al 1999). Parent-teacher agreement may be higher for more severely affected children because these children have more obvious maladaptive behaviors that they exhibit at home and at school.…”

“…Clinicians also may consider using multiple methods of assessment, reflecting both the structure of the classroom and the flexibility of the home environment. Improving the efficiency of multi-informant assessment and home-school collaboration is necessary to provide a more comprehensive, reliable depiction of social functioning that may ultimately determine the nature, cost, and intensity of interventions, and perhaps more importantly, improve outcomes for children with autism (Jepsen et al, 2012; Murray et al, 2009; Rapin et al, 1999). …”

“…Multi-informant assessment provides a richer and more accurate picture of the wide variety of social-communication and behavioral challenges that children with autism experience (Jepsen, Gray, and Taffe, 2012; Macintosh and Dissanayake, 2006, Matson and Nebel-Schwalm, 2007). Informants are expected to report on the child’s strengths and weaknesses, and subsequently prioritize related interventions (Rapin, Steinberg, and Waterhouse, 1999). …”

“…Other studies have not replicated this finding. For example, a modest correlation was found between parents and teachers on measures of behavior (Murray, Ruble, Willis, and Molloy, 2009; Rapin et al, 1999) and social skills (Murray et al, 2009) for children with autism.…”

“…This idea first emerged in the 1990’s when severity of autism was examined as a contributor to inter-rater agreement about adaptive behaviors in preschoolers (Hundert, Morrison, Mahoney, Mundy, and Vernon, 1997; Rapin et al, 1999; Voelker et al, 2000). These studies suggested that parent-teacher concordance (or lack thereof) may depend on whether children were low or high functioning.…”