Using simulations to investigate the longitudinal stability of alternative schemes for classifying and identifying children with reading disabilities

Abstract

The present study employed data simulation techniques to investigate the one-year stability of alternative classification schemes for identifying children with reading disabilities. Classification schemes investigated include low performance, unexpected low performance, dual-discrepancy, and a rudimentary form of constellation model of reading disabilities that included multiple criteria. Data from Spencer et al. (2014) were used to construct a growth model of reading development. The parameters estimated from this model were then used to construct three simulated datasets wherein the growth parameters were manipulated in one of three ways: A stable-growth pattern, a mastery learning pattern and a fan-spread pattern. Results indicated that overall the constellation model provided the most stable classifications across all conditions of the simulation, and that classification schemes were most stable in the fan-spread condition, and were the least stable under the mastery learning growth pattern. These results also demonstrate the utility of data simulations in reading research.

Phonological processing in children with specific reading disorder versus typical learners: Factor structure and measurement invariance in a transparent orthography.

“…For example, the arbitrary nature of the thresholds as well as problems due to regression to the mean are well-recognized phenomena (e.g., Francis et al, 2005;Sternberg & Grigorenko, 2002). The low diagnostic stability of current RD classification further emphasizes the limitations of the categorical approach (e.g., Brown Waesche, Schatschneider, Maner, Ahmed, & Wagner, 2011;Francis et al, 2005;Schatschneider, Wagner, Hart, & Tighe, 2016). Moreover, splitting the reading and IQ continuum produces an information loss within the established categories and may also create statistical artefacts and may reduce statistical power (e.g., Cohen, 1983;McCallum, Zhang, Preacher, & Rucker, 2002;Maxwell & Delaney, 1993).…”

Longitudinal Stability of Reading Difficulties: Examining the Effects of Measurement Error, Cut-Offs, and Buffer Zones in Identification

“…Examining the effects of measurement error with the use of observed data is not possible because measurement error is constant in observed variables and cannot be manipulated. It can be manipulated though with the use of simulations (Schatschneider et al, 2016). In addition to the use of single cut-off, we examined the stability of RD identification by using a simulation-based buffer zone in order to handle both the effects of the use of single cutoff on continuous distributions (Shankweiler et al, 1999) and the effects of the measurement error.…”

Introduction to the special issue. Advancing the state-of-the-science in reading research through modeling

“…The final paper in this section, by Schatschneider and colleagues (Schatschneider, Wagner, Hart, & Tighe, 2016) utilizes simulated data to examine the stability of various reading disability schemes over the course of a year. Currently, the lack of a clear consensus on how to most appropriately identify and classify individuals with a reading disability (RD), the relatively poor classification stability within an approach (e.g., Barth, et al, 2008; Brown Waesche, Schatschneider, Maner, Ahmed, & Wagner, 2011), and low or moderate agreement between classification approaches (Spencer, et al, 2014) leads to clearly undesirable consequences in our ability to identify children with RD.…”

“…Currently, the lack of a clear consensus on how to most appropriately identify and classify individuals with a reading disability (RD), the relatively poor classification stability within an approach (e.g., Barth, et al, 2008; Brown Waesche, Schatschneider, Maner, Ahmed, & Wagner, 2011), and low or moderate agreement between classification approaches (Spencer, et al, 2014) leads to clearly undesirable consequences in our ability to identify children with RD. Schatschneider et al(2016) conducted simulation studies to examine the likelihood that 1) measurement error, 2) regression to the mean and 3) true inter-individual differences in change of disability categorization underlie some of this longitudinal instability. Simulation data is particularly well suited to examine these issues – that is, one can manipulate simulated datasets to systematically investigate each of these three scenarios with a known “ground truth”.…”

“…We have seen demonstrations of novel analysis approaches that can reveal complex linear and nonlinear relationships among predictors of reading ability (Sideridis et al, 2016), and change over time (Schatschneider et al, 2016), as well as an application of a more well-known approach that is under-utilized in reading research in dealing with issues of collinearity and designs with large numbers of predictors common to reading research (Matsuki et al, 2016). It may be helpful, in the context of Rueckl’s (2016) contribution, to think of these approaches as refinements to our ability to identify and isolate the “control parameters” that influence individuals’ developmental trajectories, or to trace these trajectories out in more detail.…”