$L_p$ Loss Functions in Invariance Alignment and Haberman Linking

2020

Stats

Self Cite

The comparison of group means in latent variable models plays a vital role in empirical research in the social sciences. The present article discusses an extension of invariance alignment and Haberman linking by choosing the robust power loss function ρ(x)=|x|p(p>0). This power loss function with power values p smaller than one is particularly suited for item responses that are generated under partial invariance. For a general class of linking functions, asymptotic normality of estimates is shown. Moreover, the theory of M-estimation is applied for obtaining linking errors (i.e., inference with respect to a population of items) for this class of linking functions. In a simulation study, it is shown that invariance alignment and Haberman linking have comparable performance, and in some conditions, the newly proposed robust Haberman linking outperforms invariance alignment. In three examples, the influence of the choice of a particular linking function on the estimation of group means is demonstrated. It is concluded that the choice of the loss function in linking is related to structural assumptions about the pattern of noninvariance in item parameters.

“…Author Note: This article substantially extends a preprint version that appeared as "L p Loss Functions in Invariance Alignment and Haberman Linking" [187].…”

Section: Conflicts Of Interestmentioning

confidence: 77%

Lp Loss Functions in Invariance Alignment and Haberman Linking with Few or Many Groups

2020

Stats

Self Cite

“…There are also alternative linking approaches that directly rely on estimated item parameters instead of IRFs, such as mean-mean linking [17], Haberman linking based on regression modeling [27], invariance alignment [28], and distance-based measures (like χ 2 ; [29,30]), to name a few. For Haberman linking and invariance alignment, robust alternatives were recently studied [10,[31][32][33]. The linking approach is a two-step method as separate scalings are applied group-wise in the first step.…”

Section: Haebara Linkingmentioning

confidence: 99%

Robust Haebara Linking for Many Groups: Performance in the Case of Uniform DIF

2020

Psych

Self Cite

The comparison of group means in item response models constitutes an important issue in empirical research. The present article discusses a slight extension of the robust Haebara linking approach of He and Cui [Appl. Psychol. Meas., 44, 296–310, 2020] by proposing a flexible class of robust Haebara linking functions for comparisons of many groups. These robust linking functions are robust against violations of invariance. In this article, we investigate the performance of robust Haebara linking in the presence of uniform DIF effects. In an analytical derivation, it is shown that the robust Haebara linking approach provides unbiased estimates of group means in the limiting case p=0. In a simulation study, it is demonstrated that the proposed variant of the Haebara linking approach outperforms existing implementations of Haebara linking to some extent. In an empirical application using PISA data, it is illustrated that country means can be sensitive to the choice of linking functions.

On the Choice of the Item Response Model for Scaling PISA Data: Model Selection Based on Information Criteria and Quantifying Model Uncertainty

2022

Entropy

In educational large-scale assessment studies such as PISA, item response theory (IRT) models are used to summarize students’ performance on cognitive test items across countries. In this article, the impact of the choice of the IRT model on the distribution parameters of countries (i.e., mean, standard deviation, percentiles) is investigated. Eleven different IRT models are compared using information criteria. Moreover, model uncertainty is quantified by estimating model error, which can be compared with the sampling error associated with the sampling of students. The PISA 2009 dataset for the cognitive domains mathematics, reading, and science is used as an example of the choice of the IRT model. It turned out that the three-parameter logistic IRT model with residual heterogeneity and a three-parameter IRT model with a quadratic effect of the ability θ provided the best model fit. Furthermore, model uncertainty was relatively small compared to sampling error regarding country means in most cases but was substantial for country standard deviations and percentiles. Consequently, it can be argued that model error should be included in the statistical inference of educational large-scale assessment studies.