Patterns of cognitive strengths and weaknesses: Identification rates, agreement, and validity for learning disabilities identification.

Abstract

Purpose Few empirical investigations have evaluated LD identification methods based on a pattern of cognitive strengths and weaknesses (PSW). This study investigated the reliability and validity of two proposed PSW methods: the concordance/discordance method (C/DM) and cross battery assessment (XBA) method. Methods Cognitive assessment data for 139 adolescents demonstrating inadequate response to intervention was utilized to empirically classify participants as meeting or not meeting PSW LD identification criteria using the two approaches, permitting an analysis of: (1) LD identification rates; (2) agreement between methods; and (3) external validity. Results LD identification rates varied between the two methods depending upon the cut point for low achievement, with low agreement for LD identification decisions. Comparisons of groups that met and did not meet LD identification criteria on external academic variables were largely null, raising questions of external validity. Conclusions This study found low agreement and little evidence of validity for LD identification decisions based on PSW methods. An alternative may be to use multiple measures of academic achievement to guide intervention.

Accuracy and Validity of Methods for Identifying Learning Disabilities in a Response-to-Intervention Service Delivery Framework

“…Validity Miciak et al (2014b) also evaluated the external validity of the identification approaches by comparing the performance of participants that met criteria and participants that did not meet criteria on academic measures which were not utilized to identify groups. Comparisons failed to find meaningful differences in academic performance between emergent subgroups for either method.…”

“…However, all three approaches demonstrated moderate to low sensitivity and very low positive predictive power, because false-positive rates for LD identification were high . Miciak et al (2014b) utilized observed data for a sample of 139 adolescents demonstrating inadequate RTI to empirically classify participants as meeting or not meeting PSW LD identification criteria according to the C/DM and XBA methods. An investigation of the resulting identification decisions permitted an analysis of the identification rates, agreement, and external validity of the two identification approaches.…”

Can Intelligence Testing Inform Educational Intervention for Children with Reading Disability?

“…Furthermore, there was little difference in the academic performance of those who met, or did not meet, the learning disabled criteria, thus posing questions about the external validity of the PSW model. Given these findings, Miciak and his colleagues [45] concluded that it would be difficult to justify the significant resource implications that follow on from widespread use of PSW methods. A similar study, this time with second grade children [46] arrived at broadly similar conclusions.…”

“…The additional contribution provided by the cognitive measures did not appear to justify the costs involved. Recent studies [42,45,73] have similarly concluded that there is minimal value in assessing cognitive characteristics for the purpose of predicting how children will respond to intervention. Put simply, the additional information such assessments can provide, above and beyond that yielded by baseline academic assessment or curriculum based or embedded measures [74] appears not to be worth the investment of resources that is required.…”

“…Thus, cognitive profiling approaches have been criticized for questionable reliability leading to an inability to provide scientifically rigorous diagnoses [43][44][45][46]. Miciak et al (2014) [45] examined the utility of two common PSW methods, (a) the concordance/discordance method [38] and (b) the cross-battery assessment method [39]. It was found that a relatively low proportion of participants was identified as learning disabled, and there were considerably differences in who was flagged up by the two approaches.…”