Evaluating Robust Scale Transformation Methods With Multiple Outlying Common Items Under IRT True Score Equating

Abstract

Item parameter estimates of a common item on a new test form may change abnormally due to reasons such as item overexposure or change of curriculum. A common item, whose change does not fit the pattern implied by the normally behaved common items, is defined as an outlier. Although improving equating accuracy, detecting and eliminating of outliers may cause a content imbalance among common items. Robust scale transformation methods have recently been proposed to solve this problem when only one outlier is present in the data, although it is not uncommon to see multiple outliers in practice. In this simulation study, the authors examined the robust scale transformation methods under conditions where there were multiple outlying common items. Results indicated that the robust scale transformation methods could reduce the influences of multiple outliers on scale transformation and equating. The robust methods performed similarly to a traditional outlier detection and elimination method in terms of reducing the influence of outliers while keeping adequate content balance.

Lp Loss Functions in Invariance Alignment and Haberman Linking with Few or Many Groups

“…In the case of partial invariance, only a few of the pairwise differences of item parameters are nonzero. This motivates the use of robust loss functions ρ that are obtained with p ≤ 1 because a few large differences between group-specific item parameters can be interpreted as outlying cases [51][52][53][54][55][56][57][58][59]. Asparouhov and Muthén [4,24] implemented the loss function ρ(x) = |x| = |x| 0.5 in their commercial Mplus software [60].…”

Robust Haebara Linking for Many Groups: Performance in the Case of Uniform DIF

“…Linking based on the function H in Equation 2is referred to as robust Haebara linking and generalizes the originally proposed Haebara linking method for two groups [3] that uses the loss function ρ(x) = x 2 . He and colleagues [4,22] considered the loss function ρ(x) = |x| for two groups. Haebara linking for multiple groups was investigated in several articles [10,[23][24][25].…”

“…There is no down-weighting of large DIF effects because the weights only involve the integrated information functions. In the case of p = 1 (as proposed in [4,22]), it can be shown that the bias in estimated group means in robust Haebara linking is a weighted median of DIF effects (see Equation (A15) in Appendix A.5 and [39]).…”

“…In this article, we investigate robust variants to the originally proposed Haebara linking method [3] for many groups. We study a slight extension of robust Haebara linking that was proposed by He and Cui [4] by using a more flexible class of loss functions. We use a two-parameter logistic model (2PL) item response model to introduce the methodology.…”

$L_p$ Loss Functions in Invariance Alignment and Haberman Linking

“…In the case of partial invariance, only a few of the pairwise differences of item parameters are nonzero. This motivates the use of robust loss functions ρ because a few large differences can be interpreted as outlying cases [38][39][40][41][42][43][44][45][46]. Asparouhov and Muthén [4,21] implemented the loss function ρ(x) = |x| = |x| 0.5 in their commercial Mplus software [47].…”