Evaluating item response theory linking and model fit for data from PISA 2000–2012

Robust Haebara Linking for Many Groups: Performance in the Case of Uniform DIF

“…Biased items are allowed to possess DIF effects e ig = 0, which differs from zero. This situation is also referred to in the literature as partial invariance [11,12]. If there are no biased items, all item parameters are invariant, which is denoted as full invariance.…”

“…Alternatively, biased items could be determined by a statistical DIF detecting method prior to linking (see, e.g., [12,20,21]). The linking method is then subsequently applied only on the anchor items.…”

Changing times, changing needs: enhancing the utility of international large-scale assessments

“…On occasion, a new scale is defined and the items from previous administrations are placed on this new scale. In either case, the relevant procedures are well known, as are the checks on the validity of the linkage (Mazeo and von Davier 2014;von Davier et al 2019). With regard to the background questionnaire, the chief requirement is to retain the item set contributing to the composite scale, or-if changes are absolutely necessary-to make only minimal alterations.…”

“…12 In any case, these more complex designs require more sophisticated statistical/psychometric models to carry out the calculations yielding the plausible values that are the basis for reporting and analysis. Because the technical challenges are considerable, measurement specialists are devising new strategies for carrying out differential item functioning analyses, as well as better measures of general model fit with accompanying diagnostics (Yamamoto, Shin et al 2018;Yamamoto, Khorramdel et al 2018;Chen et al 2014;von Davier et al 2019).…”

Invariance analyses in large-scale studies

“…In practice, the ratings observed for the hypothetical individuals often violate the expected ratings, particularly in low-ability groups (perhaps due to respondent disengagement, or to the high cognitive load that this procedure imposes to participants). And some of the purported "improvements" in reliability and validity may be artificial, simply due to mathematical properties of the method, rather than to the substantive information gained about the response style of individuals (von Davier et al, 2018 [121]). Furthermore, the choice of the vignettes appears not to be neutral with respect to the substantive conclusions (Stankov, Lee and von Davier, 2018 [122]).…”