Reliability of Decision-Making Frameworks for Response to Intervention for Reading

Abstract

The current study examines the consistency of two response-to-intervention (RTI) decision-making models. Weekly progress monitoring data for 30 students participating in a Tier II intervention were collected for 30 weeks. The data were examined by comparing them to an aimline with a yearly goal and by computing a dual discrepancy (DD) using numerical slope and postintervention reading level. A κ coefficient was computed between the two models regarding student growth using three categorical decisions of making sufficient progress, not making sufficient progress, and exceeding expected progress. The resulting coefficient of .29 represents fair agreement. Data were also assigned to one of two sets in an every-other order, with resulting new aimlines and new DDs. The consistency between the two sets of data resulted in coefficients of κ = .29 for aimline and κ = .58 for DD. Finally, the standard error of measurement for the baseline data point affected the decision made (κ = .13) for aimline but not for DD (κ = .93). Implications for research, practice, and special education decision making are included.

Accuracy and Validity of Methods for Identifying Learning Disabilities in a Response-to-Intervention Service Delivery Framework

“…The studies above documented poor to fair agreement even when the same method to determine intervention response was utilized (Barth et al 2008;Brown-Waesche et al 2011) and poor internal consistency for identification decisions (Burns et al 2010). Poor agreement is also documented in studies that evaluate different ways to operationalize a single method, such as a dual-discrepancy method.…”

“…Similar to Barth et al (2008), agreement was higher for identifying adequate responders with lower agreement for determinations of inadequate response. Burns et al (2010) evaluated agreement and internal consistency for decisions about the adequacy of response using a dual-discrepancy method and a method based on progress toward a final benchmark (aim-line approach). The two methods were applied to determine if students were making inadequate progress, adequate progress, or exceeding expectations.…”

The Impact of Baseline Trend Control on Visual Analysis of Single-Case Data

“…Consequently, graphical methods of baseline trend control such as MASAJ should be investigated as visual aids that could potentially improve decision making in the context of RTI. Current methods to determine responsiveness to intervention include the aimline technique (i.e., comparing the number of data points above a line drawn between the first point and a desired level of performance) and the dual discrepancy approach (i.e., comparing post-intervention scores to an absolute level of performance and the student's rate of growth to typical rates of growth), with decisions based on the dual discrepancy approach appearing to be more reliable (Burns, Scholin, Kosciolek, & Livingston, 2010). Given that baseline trend could also impact decisions of responsiveness, the effects of baseline trend control in combination with these methods of determining responsiveness should be investigated.…”

Agreement and Coverage of Indicators of Response to Intervention

“…Only a few studies report the actual levels of agreement and coverage across methods derived from RTI (Barth et al, 2008; Brown Waesche, Schatschneider, Maner, Ahmed, & Wagner, 2011; Burns, Scholin, Kosciolek, & Livingston, 2010; Speece & Case, 2001). These studies show levels of agreement in the poor to fair range that are generally below desirable levels.…”