Expected Equating Error Resulting From Incorrect Handling of Item Parameter Drift Among the Common Items

Abstract

Incorrect handling of item parameter drift during the equating process can result in equating error. If the item parameter drift is due to construct-irrelevant factors, then inclusion of these items in the estimation of the equating constants can be expected to result in equating error. On the other hand, if the item parameter drift is related to the construct being measured, then removal of those items from the estimation of the equating constants can be expected to result in equating error. The effect of incorrect handling of item parameter drift on equating has been examined via simulation studies and other methods by a number of authors with mixed results. In this article, this effect is assessed by estimating the expected equating error under the assumptions that the three-parameter item response theory model is used for calibration—equating.

Modeling local dependence in longitudinal IRT models

“…This is often called item parameter drift (Wells et al, 2002;Miller & Fitzpatrick, 2009) and the assumption that item parameters are stable over time should also be tested. Different methods for detection of item parameter drift exist (Donoghue & Isham, 1998;DeMars, 2004;Galdin & Laurencelle, 2010).…”

“…Another key assumption when measuring trends over time is that the item parameters do not change over time (Wells et al, 2002;Miller and Fitzpatrick 2009). This can be considered differential item functioning with respect to time and is often referred to as item parameter drift.…”

The Philosophical Aspects of IRT Equating: Modeling Drift to Evaluate Cohort Growth in Large‐Scale Assessments

“…Discard items based on the statistical degree of difference between two tests, up to the point of the imposed restriction of no more than 20% of the total common items. Keep perceived construct‐relevant outliers in the linking item pool, as is being done by some large‐scale assessments using Huynh's (2009) criteria. Eliminate outliers from the test based on nonconstruct related issues such as exposure, cheating, and so on because the inclusion of these items would bias the test in favor of some students, and thus jeopardize the validity of the test. Keep construct‐irrelevant outliers in the test but eliminate them from the common pool of linking items. This would pertain to common items that would have had changes, say, in the type of font used, item location change, and other similar construct‐irrelevant issues (Miller & Fitzpatrick, 2009) that are not a threat to the validity of the test. These items can now be considered as unique items.…”

“…Keep construct‐irrelevant outliers in the test but eliminate them from the common pool of linking items. This would pertain to common items that would have had changes, say, in the type of font used, item location change, and other similar construct‐irrelevant issues (Miller & Fitzpatrick, 2009) that are not a threat to the validity of the test. These items can now be considered as unique items.…”

Augmenting the use of the Rasch model under time constraints

“…The Rasch model (Rasch 1960) is now widely used in education (Bruniges 2001;Miller et al 2005). One of the main advantages it has over 'classical' or 'traditional' test theory methods is that it defines a scale for an underlying latent variable that is measured by the test items, which means that comparable measures may be computed for examinees who did not answer the same questions, without intermediate equating steps.…”