Lp Loss Functions in Invariance Alignment and Haberman Linking with Few or Many Groups

Abstract

The comparison of group means in latent variable models plays a vital role in empirical research in the social sciences. The present article discusses an extension of invariance alignment and Haberman linking by choosing the robust power loss function ρ(x)=|x|p(p>0). This power loss function with power values p smaller than one is particularly suited for item responses that are generated under partial invariance. For a general class of linking functions, asymptotic normality of estimates is shown. Moreover, the theory of M-estimation is applied for obtaining linking errors (i.e., inference with respect to a population of items) for this class of linking functions. In a simulation study, it is shown that invariance alignment and Haberman linking have comparable performance, and in some conditions, the newly proposed robust Haberman linking outperforms invariance alignment. In three examples, the influence of the choice of a particular linking function on the estimation of group means is demonstrated. It is concluded that the choice of the loss function in linking is related to structural assumptions about the pattern of noninvariance in item parameters.

Mean Comparisons of Many Groups in the Presence of DIF: An Evaluation of Linking and Concurrent Scaling Approaches

“…3 By distinguishing between reference items and biased items, we highlight the vital role that identification constraints play in estimating the means of the group-specific ability distribution. However, at a more conceptual level, it needs to be emphasized that the decision about whether an item with a DIF effect is classified as a reference item or as a biased item should not be based solely on statistical criteria (see Camilli, 1993;Gomez-Benito et al, 2018;Penfield & Camilli, 2007;Zwitser et al, 2017, for this argument). More specifically, the identification of a mean for group g relies only on items from the set of reference items J R;g and does not rely on the set J B;g of biased items.…”

“…More specifically, the identification of a mean for group g relies only on items from the set of reference items J R;g and does not rely on the set J B;g of biased items. Consequently, basing the decision to remove items from the reference item set exclusively on statistical criteria could result in construct underrepresentation (i.e., removing items with DIF effects that are construct relevant; see Camilli, 1993;Penfield & Camilli, 2007). If a researcher is confident that all items in the test are construct Mean Comparisons of Many Groups in the Presence of DIF relevant, no items with DIF effects should be removed from linking.…”

“…In this approach, country comparisons are based on a multiple-group IRT model in which, for some of the items, item-by-group interactions are specified (Oliveri & von Davier, 2011;von Davier et al, 2019). The decision about which item parameters obtain group-specific item parameters is based on test statistics for DIF (see Penfield & Camilli, 2007, for an overview). The goal is to determine the set of biased items J B;g for each group that obtain group-specific item parameters while the DIF effects in the set of reference items J R;g are set to zero.…”

“…A major methodological challenge of these comparisons is that the items of the achievement tests show differential item functioning (DIF) in specific countries. Uniform DIF is present if an item is relatively easier or more difficult in a specific country than at the international level (Holland & Wainer, 1993;Penfield & Camilli, 2007). In this case, an item parameter is noninvariant across countries.…”

Can We Compare Solidarity Across Europe? What, Why, When, and How to Assess Exact and Approximate Equivalence of First- and Second-Order Factor Models

“…However, when evaluating the simulations concerning a cross-national case with 25 groups, Pokropek et al (2020) show that the deviations between the software packages are quite close, while for different settings, like for example small number of groups, the algorithm used in the sirt package seems to perform the alignment better. In addition to carefully read the differences between the two software packages and the implications for the analyses (Pokropek et al, 2020;Robitzsch, 2020), R-users could find helpful the tutorial for measurement invariance by Fischer and Karl (2019), which also includes invariance alignment.…”