What do daily behavior report cards (DBRCs) measure? An initial comparison of DBRCs with direct observation for off-task behavior

Abstract

This study investigated the similarity of information provided from a daily behavior report card (DBRC) as rated by the teacher to direct observation data obtained from external observers. In addition, the similarity of ratings was compared with variations of problem severity (mild, severe) and teacher training (none, some). Results suggested a moderate association between teacher perceptions of behavior as measured by DBRC ratings and direct observation conducted by an external observer. In addition, 23-45% of the variance in DBRC ratings was consistent with the direct observation data. Severity of the behavior problem or the inclusion of training was not found to significantly affect the similarity of ratings. In summary, results tentatively suggest that the DBRC may be a viable supplement to direct observation for estimating behavior in applied settings. Limitations, future research directions, and implications are discussed. © 2005 Wiley Periodicals, Inc.Teacher ratings of behavior have become an integral part of the evaluation of children's behavior, with some suggestion that such ratings are the most widely used method for assessing classroom conduct problems (Hoge & Andrews, 1992). The types of teacher ratings range from norm-referenced measures of a broad range of behaviors to educator-made questions about a narrower set of classroom behaviors. The latter, which we refer to as a daily behavior report card (DBRC), serves as the focus of the current investigation. A measure is considered a DBRC if a specified behavior is rated at least daily, and that information is shared with someone other than the rater (Chafouleas, Riley-Tillman, & McDougal, 2002). Although various other terms have been used to describe a DBRC when used as an intervention tool (e.g., good behavior note, home school note, daily report card), using such measures as a way to collect information about student behavior may be considered a type of performance-based behavioral recording. In such a performance-based behavior recording procedure, behaviors are identified and described, and then a Likert-type scale is developed that corresponds to specific dimensions of behavior (Steege, Davin, & Hathaway, 2001).Historically, systematic direct observation has been considered an essential component of behavioral assessment (Alessi, 1998;Hintze, Volpe, & Shapiro, 2002). It has been argued that direct observation, although not without limitations, lends itself to more precise (i.e., reliable and accurate) estimates of behavior because the information is collected as the behavior actually occurs.

Preliminary evaluation of various training components on accuracy of Direct Behavior Ratings

“…That is, participants in the ST condition read for a total of 20 minutes, whereas participants in the FOR + RET-6 condition did not read at all. Although extant research would suggest that indirect procedures such as reading a behavior assessment text may not be enough to result in improved performance (e.g., Chafouleas et al, 2005;Sterling-Turner et al, 2001), the assumption that reading an assessment text is a neutral within-condition activity is not beyond reproach. Finally, although the brief viewing and rating of video clips of classroom behavior presents an efficient option for accomplishing study goals, generalization to actual classroom settings cannot be inferred.…”

“…Researchers compared these ratings with teacher ratings on the same tool as well as direct observation data, and results also failed to find a strong correspondence between DBR and SDO recordings. With regard to DBR-SIS, one preliminary study did not find support for hybrid indirect training procedures on rater accuracy (Chafouleas, McDougal, Riley-Tillman, Panahon, & Hilt, 2005). In that study, teachers were assigned to conditions involving either no training or indirect training involving reading vignettes, practicing rating of off-task behavior, and then comparing their rating with pre-determined responses.…”

Bringing research into practice to intervene with young behaviorally challenging students in public school settings: Evaluation of the behavior consultation team (BCT) project

“…Daily Behavior Report Cards (DBRCs) are performance-based recording procedures used to collect teacher ratings of a student specific to a predetermined set of behaviors (Chafouleas et al, 2002). Given the applied nature of this investigation, DBRCs were selected for intervention monitoring based on prior research suggesting that these procedures are feasible and acceptable for use in schools, effective in promoting positive student behavior, useful for increasing home school communication, and reasonably correlated with direct observation results (Chafouleas, McDougal, Riley-Tillman, Panahon, & Hilt, 2004;Chafouleas et al, 2002).…”

Tier II Interventions within the Framework of School-Wide Positive Behavior Support: Essential Features for Design, Implementation, and Maintenance

“…Although office discipline referral patterns might be one indicator of overall effects of a Tier II intervention within a school, they are unlikely to be useful for progress monitoring because they do not provide frequent and repeated measurement. Daily report cards (Chafouleas, McDougal, Riley-Tillman, Panahon, & Hilt, 2005;Chafouleas, Riley-Tillman, & Sassu, 2006) have promise in this area, but have been used primarily for interventions in place across the school day. Research is needed to determine whether daily report cards can be used to monitor a wider variety of interventions, including those that occur for brief periods of time during a school day (such as social skills groups).…”