Person Ability Scores as an Alternative to Norm-Referenced Scores as Outcome Measures in Studies of Neurodevelopmental Disorders

Abstract

Although norm-referenced scores are essential to the identification of disability, they possess several features which affect their sensitivity to change. Norm-referenced scores often decrease over time among people with neurodevelopmental disorders who exhibit slower-than-average increases in ability. Further, the reliability of norm-referenced scores is lower at the tails of the distribution, resulting in floor effects and increased measurement error for people with neurodevelopmental disorders. In contrast, the person ability scores generated during the process of constructing a standardized test with item response theory are designed to assess change. We illustrate these limitations of norm-referenced scores, and relative advantages of ability scores, using data from studies of autism spectrum disorder and creatine transporter deficiency.

Measuring Depression in Autistic Adults: Psychometric Validation of the Beck Depression Inventory–II

“…IRT-based latent trait scores are also associated with different standard error estimates at each point along the latent trait continuum, allowing score reliability to be estimated for each individual separately and providing more accurate score confidence intervals (CIs). Despite these and other advantages, IRT-based measurement tools are largely unused in autism research and clinical practice (though see Farmer et al, 2020). One major obstacle preventing the widespread use IRT-based scoring in these settings is that many clinicians and researchers lack the specific knowledge and expertise needed to calculate IRT-based latent trait scores from published item parameters.…”

Use of the Vineland‐3, a measure of adaptive functioning, in CLN3

“…Similar to the SIB-R, Vineland-3 is amenable to data collection from adult caregivers and includes norm references for infants to adults with broad ability range (Fletcher, 2014). In addition, the Vineland-3 offers the ability to collect data remotely, is more amenable to collection of data from non-English speaking participants (where interpreters were able to be used), and includes person ability scores, which directly indicate changes in ability rather than relative to age-based expectations (Farmer et al, 2020). This latter scoring option addresses the limitation of standardized scores to track stabilization or improvement in a pediatric neurodegenerative process (Farmer et al, 2020;Kronenberger et al, 2021).…”

“…In addition, the Vineland-3 offers the ability to collect data remotely, is more amenable to collection of data from non-English speaking participants (where interpreters were able to be used), and includes person ability scores, which directly indicate changes in ability rather than relative to age-based expectations (Farmer et al, 2020). This latter scoring option addresses the limitation of standardized scores to track stabilization or improvement in a pediatric neurodegenerative process (Farmer et al, 2020;Kronenberger et al, 2021). Results from this study showed that adaptive behavior scores obtained from the Vineland-3 should be considered as an outcome measure for neurodevelopmental functional assessment in CLN3, as its scores correlate strongly with UBDRS and VIQ scores.…”

“…Cognitive performance is a common outcome (Adams et al, 2013;Augustine et al, 2019;Lamminranta et al, 2001), but at least a portion of the IQ tests used to measure cognition are invalidated by vision loss. Frequently used IQ tests have decreased sensitivity in the lower range (Farmer et al, 2020), limiting their use in the setting of progressive neurocognitive decline in CLN3. Adaptive behavior, the collection of skills required for age-appropriate functioning in one's environment, is correlated but not redundant with cognitive performance (Bolte & Poustka, 2002;Freeman et al, 1988;Hayes & Farnill, 2003;Keith et al, 1987).…”

“…For current therapeutic approaches aimed at slowing, rather than reversing, neurodegeneration, and outcome measures that enable meaningful longitudinal comparisons of nonstandardized scores (e.g., person ability (Farmer et al, 2020)) would be more applicable for trials.…”