Monte Carlo Modeling of Contemporary Intelligence Test (IQ) Factor Structure: Implications for IQ Assessment, Interpretation, and Theory

Abstract

Researchers continue to debate the constructs measured by commercial ability tests. Factor analytic investigations of these measures have been used to develop and refine widely adopted psychometric theories of intelligence particularly the Cattell–Horn–Carroll (CHC) model. Even so, this linkage may be problematic as many of these investigations examine a particular instrument in isolation and CHC model specification across tests and research teams has not been consistent. To address these concerns, the present study used Monte Carlo resampling to investigate the latent structure of four of the most widely used intelligence tests for children and adolescents. The results located the approximate existence of the publisher posited CHC theoretical group factors in the Differential Abilities Scales–Second edition and the Kaufman Assessment Battery for Children–Second edition but not in the Wechsler Intelligence Scale for Children–Fifth edition or the Woodcock–Johnson IV Tests of Cognitive Abilities. Instead, the results supported alternative conceptualizations from independent factor analytic research. Additionally, whereas a bifactor model produced superior fit indices in two instruments (Wechsler Intelligence Scale for Children–Fifth edition and Woodcock–Johnson IV Tests of Cognitive Abilities), a higher order structure was found to be superior in the Kaufman Assessment Battery for Children–Second edition and the Differential Abilities Scales–Second edition. Regardless of the model employed, the general factor captured a significant portion of each instrument’s variance. Implications for IQ test assessment, interpretation, and theory are discussed.

Factor Structure of the Differential Ability Scales–Second Edition Core Subtests: Standardization Sample Confirmatory Factor Analyses

“…The present CFA results reinforce the admonition of extreme caution for any interpretations of DAS-II scores beyond the GCA (Canivez & McGill, 2016; Dombrowski, Golay, et al, 2018, 2019), including assessments for PSW. Due to the very small portions of unique true score variance provided by cluster scores and the inability to locate the NV score consistently across the age span of the test, such scores and their comparisons are potentially misleading.…”

Factor Analysis and Variance Partitioning in Intelligence Test Research: Clarifying Misconceptions

“…This is also evident in the fit statistics reported in their Table 7, which indicated that Model 1, in their analyses, is structurally deficient and should not be retained. Inadequacy of their Model 1 ( g only) was also reported in every CFA study reported in Decker et al’s Table 1 and studies conspicuously absent from that systematic review (e.g., Canivez, Watkins, & Dombrowski, 2017; Canivez, Watkins, Good, et al, 2017; Canivez, Watkins, & McGill, 2019; Dombrowski, McGill & Morgan, 2019; Fenollar-Cortes & Watkins, 2019; Lecerf & Canivez, 2018). In sum, it is unclear what the Decker et al CFA results prove relative to determining whether the BF model is biased or the psychometric adequacy of the variance partitioning procedures that emanate from that model.…”

“…We also draw attention to Decker and colleagues omission of research that has supported the retention of a higher-order (HO) model at the expense of the BF model (e.g., Dombrowski, McGill, & Morgan, 2019; McGill, 2020) and other research that has impartially presented both the HO and BF models to readers when they could not be distinguished statistically (e.g., Canivez, Watkins, & Dombrowski, 2017; Canivez, Watkins, Good, James, & James, 2017; Dombrowski, Golay, McGill, & Canivez, 2018; Strickland, Watkins, & Caterino, 2015). Further, Decker et al also overlooked studies that have acknowledged the existence of broad abilities beyond g (e.g., Benson, Beaujean, McGill, & Dombrowski, 2018).…”

The Validity of WISC-V Profiles of Strengths and Weaknesses

“…The appropriateness of the five factor model is important as it underlies the index scores which form the basis for the interpretation of strengths and weaknesses in cognitive abilities. The five factor model reported in the technical manual of the American version has redundancies as shown by re-analyses of data (e.g., Dombrowski et al, 2015; Dombrowski et al, 2019) as well as by a simulation study (Dombrowski, et al, 2021). For example, the second order g factor and the broad factor Fluid Reasoning are identical and the model implied correlation of Fluid Reasoning with Visual Spatial is .88.…”