Contextual Explanations of Local Dependence in Item Clusters in a Large Scale Hands‐On Science Performance Assessment

Abstract

This study provides hypothesized explanations for local item dependence (LID) in a large scale hands‐on science performance assessment. Items within multi‐step item clusters were classified as low or high in LID using contextual analysis procedures described in this and other studies. LID was identified statistically using the average within cluster (AWC) correlation procedure described in previous studies. Levels of LID identified in contextual analyses were compared to levels of LlD identified in correlation analyses. Consistent with other studies, items that appear to elicit locally dependent responses require examinees to answer and explain their answer or to use given or generated information to respond

On the need for negative local item dependence

“…They also paraphrase Rosenbaum and claim that he shows that it is possible for an exam to have simply positive local dependencies between items, with no negative ones. Other examples that may be worthy of extra attention include : Ferrara, Huynh, and Michaels (1999) where the negative dependencies found between the clusters are dismissed as statistical artifacts, Ip (2000) which attempts to estimate the loss of information due to item clusters using only the positive item associations, and Jannerone's (1997) conjunctive IRT model which attempts to fit Rosenbaum's positive item-bundle model without resorting to polytomous item models.…”

A Speeded Item Response Model with Gradual Process Change

“…We refer to Bradlow, Wainer and Wang (1999) and Tuerlinckx and De Boeck (2004) for possible approaches to modelling local item dependencies. Yen (1993) and Ferrara, Huynh and Michaels (1999) provide a detailed taxonomy of possible reasons for the existence of local item dependency. One of the most prevalent causes in educational testing is test speededness.…”

A Boundary Mixture Approach to Violations of Conditional Independence

“…In practice some subsets of items appeal to the same specific background theme, use the same stimulus material, are subquestions of the same problematic case, or in other 58 PSYCHOMETRIKA words, share some common ground that is not directly relevant to the more general construct underlying the whole test. It can be expected that responses on such subset items will partially show dependence due to their shared idiosyncratic features and not only because they relate to the skill or ability intended to be measured by the test as a whole (Ferrara, Huynh, & Michaels, 1999). Such residual local item dependencies (LID) indicate that the model fails to correctly account for the item dependence structure, resulting in an unwanted negative impact on model results and related inferences.…”