Improving measurement-invariance assessments: correcting entrenched testing deficiencies

Abstract

BackgroundFactor analysis historically focused on measurement while path analysis employed observed variables as though they were error-free. When factor- and path-analysis merged as structural equation modeling, factor analytic notions dominated measurement discussions – including assessments of measurement invariance across groups. The factor analytic tradition fostered disregard of model testing and consequently entrenched this deficiency in measurement invariance assessments.DiscussionApplying contemporary model testing requirements to the so-called configural model initiating invariance assessments will improve future assessments but a substantial backlog of deficient assessments remain to be overcome.This articlesummarizes the issues,demonstrates the problem using a recent example,illustrates a superior model assessment strategy,and documents disciplinary entrenchment of inadequate testing as exemplified by the journal Organizational Research Methods. SummaryEmploying the few methodologically and theoretically best, rather than precariously-multiple, indicators of latent variables increases the likelihood of achieving properly causally specified structural equation models capable of displaying measurement invariance. Just as evidence of invalidity trumps reliability, evidence of configural model misspecification trumps invariant estimates of misspecified coefficients.Electronic supplementary materialThe online version of this article (doi:10.1186/s12874-016-0230-3) contains supplementary material, which is available to authorized users.

Work Engagement among Rescue Workers: Psychometric Properties of the Portuguese UWES

“…The measurement invariance corresponds to the situation where links between the items and the construct studied are invariant for different contexts, such as countries, religions, times, or languages (Gregorich, 2006 ; Little, 2013 ; Brown, 2015 ; Hayduk, 2016 ). This should be tested on the construction or adaptation plan of any psychometric instrument (Edwards et al, 2017 ).…”

Construct Validity of the WISC-V in Clinical Cases: Exploratory and Confirmatory Factor Analyses of the 10 Primary Subtests

“…Reynolds and Keith (2017) also investigated the measurement invariance of the WISC-V across age groups with CFA, but only examined an oblique five-factor model, which did not include a general intelligence dimension. As noted by Hayduk (2016), if the number of factors are not accurately specified then “asking about invariance between groups is asking whether the groups agree in their mis representation of the connections between the indicators and the underlying latent variables” (p. 2).…”

“…Reynolds and Keith (2017) also investigated the measurement invariance of the WISC-V across age groups with CFA, but only examined an oblique five-factor model, which did not include a general intelligence dimension. As noted by Hayduk (2016), if the number of factors are not accurately specified then "asking about invariance between groups is asking whether the groups agree in their misrepresentation of the connections between the indicators and the underlying latent variables" (p. 2). Reynolds and Keith (2017) also explored numerous (perhaps post hoc) model modifications for five-factor firstorder models and then for both higher-order and bifactor models including five group factors to better understand WISC-V measurement.…”

Evaluation of Latent Models Assessing Physical Fitness and the Healthy Eating Index in Community Studies: Time-, Sex-, and Diabetes-Status Invariance

“…Accurate measurement and representation of summary indices and measures requires assessment of measurement equivalence/invariance (ME/I) to demonstrate that the items/tests/measurements perform equally well and measure the same underlying constructs across groups and/or over time [39][40][41][42][43]. The longitudinal design of the current study allows for analysis of a combination of cross sectional and change models, as well as assessment of ME/I [41].…”

“…Information garnered from cross-sectional models and the evaluation of appropriate measurement models developed through EFA and CFA methods will help inform models to be used to predict changes within constructs as well as testing structural relationships between constructs. As noted by Hayduk [39] creation of latent variables relies on accurate measurement of observed constructs.…”