Exploring the Full-Information Bifactor Model in Vertical Scaling With Construct Shift

Abstract

To address the lack of attention to construct shift in IRT vertical scaling, a bifactor model is proposed to estimate the common dimension for all grades and the grade-specific dimensions. The bifactor model estimation accuracy is evaluated through a simulation study with manipulated factors of percent of common items, sample size, and degree of construct shift. In addition, the unidimensional IRT (UIRT) estimation model that ignores construct shift is examined to represent the current practice for IRT vertical scaling; comparisons on parameter estimation accuracy of the bifactor and UIRT models are discussed.The major findings of the simulation study are (1) bifactor models are well recovered overall, even though item discrimination parameters are underestimated to a small degree; (2) item discrimination parameter estimates are overestimated in UIRT models due to the effect of construct shift; (3) person parameters of UIRT models are less accurately estimated than that of bifactor models, and the accuracy decreases as the degree of construct shift increases; (4) group mean parameter estimates of UIRT models are less accurate than that of bifactor models, and a large effect due to construct shift is found for the group mean parameter estimates of UIRT models.The real data analysis provides an illustration of how bifactor models can be applied to a problem involving for vertical scaling with construct shift. General procedures for testing practice are also discussed.

IRT Model Misspecification and Measurement of Growth in Vertical Scaling

“…Yen () associated such complexity with increased multidimensionality. This explanation is consistent with a frequently‐cited concern with vertical scaling, namely that the unidimensionality assumption is violated (Li & Lissitz, ; Martineau, ). However, as noted by Lord (), increases in item complexity across grades can also lead to shrinkage problems in a unidimensional framework when understood in relation to conjunctively interacting item components, such as represented by the LPE.…”

“…Vertical scaling is one context in which IRT model fit is important to evaluate. Several studies have attended to multidimensionality as a source of model misfit in vertical scaling (Li & Lissitz, 2012;Martineau, 2006;Wang & Jiao, 2009); however, functional form misfit (i.e., misspecification of unidimensional item characteristic curves [ICCs]) is likely also of great concern due to the strong invariance assumptions made when placing groups of different ability distributions (such as occur across grades) on a common metric. For example, it has been seen in differential item functioning (DIF) applications that functional form misfit, even when small, can lead to false rejections of item parameter invariance when the groups being compared differ only in ability (Bolt, 2002;Shepard, Camilli, & Williams, 1984).…”

The Rediscovery of Bifactor Measurement Models

“…It only is recently that bifactor models have been rediscovered as an important alternative structural representation of multidimensionality and a topic of research and application in item response theory (IRT) and structural equation modeling (SEM). Evidence of this renewed enthusiasm is abundant and comes in several forms, for example: Major personality and assessment journals now routinely include articles demonstrating applications of bifactor modeling (e.g., Ackerman, Donnellan, & Robins, 2012; Bados, Gomez-Benito, & Balaguer, 2010; Ebesutani et al, in press; Gibbons, Ruch, & Immekus, 2009; Gignac, Palmer, & Stough, 2007; Patrick, Hicks, Nichol, & Krueger, 2007);Didactic articles recently have appeared arguing for the utility of bifactor models in resolving important problems in conceptualizing and measuring psychological constructs (Brunner, Nagy, & Wilhelm, in press; Cai, Yang, & Hansen, 2011; Chen, Hayes, Carver, Laurenceau, & Zhang, 2012; Gustafsson & Aberg-Bengtsson, 2010; Reise, Moore, & Haviland, 2010; Reise, Morizot, & Hays, 2007; Thomas, 2012);Published psychometric articles now compare the bifactor to competing structural representations (Chen, West, & Sousa, 2006; Rijmen, 2010), provide solutions to challenging estimation problems (Cai, 2010c; Rijmen, 2009), demonstrate important extensions of bifactor modeling to computerized adaptive testing (Gibbons, et al, 2008), vertical scaling (Li & Lissitz, 2012), and assessing differential item functioning (Fukuhara & Kamata, 2011; Jeon, Rijmen, & Rabe-Hesketh, in press); and importantly,User friendly software now is available that facilitates the estimation of parameters for a variety of latent variable models, including bifactor (e.g., IRTPRO 2.1, Cai, Thissen, & du Toit, 2011; EQSIRT , Wu & Bentler, 2011). …”

“…Published psychometric articles now compare the bifactor to competing structural representations (Chen, West, & Sousa, 2006; Rijmen, 2010), provide solutions to challenging estimation problems (Cai, 2010c; Rijmen, 2009), demonstrate important extensions of bifactor modeling to computerized adaptive testing (Gibbons, et al, 2008), vertical scaling (Li & Lissitz, 2012), and assessing differential item functioning (Fukuhara & Kamata, 2011; Jeon, Rijmen, & Rabe-Hesketh, in press); and importantly,…”

Two IRT Fixed Parameter Calibration Methods for the Bifactor Model

“…The reason for applying the two FPC methods to the bifactor model is twofold: (a) the bifactor model is flexible enough to represent structures that are commonly found in educational and psychological measurement; and (b) only a series of two-dimensional integrals needs to be evaluated regardless of the number of factors in the model. In educational measurement, the bifactor model has been applied to a variety of areas, including calibration of testlet-based tests (DeMars, 2006), vertical scaling (Li & Lissitz, 2012), differential item functioning (Jeon, Rijmen, & Rabe-Hesketh, 2013), multiple-group analysis (Cai et al, 2011), and test equating (Lee & Lee, 2016;Lee et al, 2015).…”