Evaluating Scoring Procedures for Context-Dependent Item Sets1

Keller, Lisa A.; Swaminathan, Hariharan; Sireci, Stephen G.

doi:10.1207/s15324818ame1603_3

Cited by 23 publications

(13 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Due to the content of large scale tests, studies mostly focused on passage dependency among these factors causing local item dependency; and it was identified in the conducted studies that local dependency was caused because the items were based on a common passage (Keller, Swaminathan, & Sireci, 2003;Lee, 2004;Sireci, Thissen, & Wainer, 1991;Thissen et al, 1989;Yen, 1993;Zenisky, Hambleton, & Sireci, 2002). If there is more than one item depending on the same passage LID will occur.…”

Section: Introductionmentioning

confidence: 99%

Examination of Different Item Response Theory Models on Tests Composed of Testlets

Koğar¹,

Kelecioğlu²

2017

JEL

View full text Add to dashboard Cite

The purpose of this research is to first estimate the item and ability parameters and the standard error values related to those parameters obtained from Unidimensional Item Response Theory (UIRT), bifactor (BIF) and Testlet Response Theory models (TRT) in the tests including testlets, when the number of testlets, number of independent items, and sample size change, and then to compare the obtained results. Mathematic test in PISA 2012 was employed as the data collection tool, and 36 items were used to constitute six different data sets containing different numbers of testlets and independent items. Subsequently, from these constituted data sets, three different sample sizes of 250, 500 and 1000 persons were selected randomly. When the findings of the research were examined, it was determined that, generally the lowest mean error values were those obtained from UIRT, and TRT yielded a mean of error estimation lower than that of BIF. It was found that, under all conditions, models which take into consideration the local dependency have provided a better model-data compatibility than UIRT, generally there is no meaningful difference between BIF and TRT, and both models can be used for those data sets. It can be said that when there is a meaningful difference between those two models, generally BIF yields a better result. In addition, it has been determined that, in each sample size and data set, item and ability parameters and correlations of errors of the parameters are generally high.

show abstract

Section: Introductionmentioning

confidence: 99%

Examination of Different Item Response Theory Models on Tests Composed of Testlets

Koğar¹,

Kelecioğlu²

2017

JEL

View full text Add to dashboard Cite

show abstract

“…Results under the first approach (the dichotomous IRT approach) simply indicate the robustness of traditional IRT models to LID. Plenty of research has shown that the dichotomous IRT approach could lead to misestimation of item parameters and test reliability (Keller, Swaminathan, & Sireci, 2003;Lawrence, 1995;Sireci, Thissen, & Wainer, 1991;Zenisky, Hambleton, & Sireci, 2002). However, very few studies have focused on examining the impact of LID on numbercorrect score equating when traditional IRT models are applied.…”

mentioning

confidence: 99%

Effect of Item Response Theory (IRT) Model Selection on Testlet‐Based Test Equating

Cao

Tao

2014

ETS Research Report Series

View full text Add to dashboard Cite

The local item independence assumption underlying traditional item response theory (IRT) models is often not met for tests composed of testlets. There are 3 major approaches to addressing this issue: (a) ignore the violation and use a dichotomous IRT model (e.g., the 2‐parameter logistic [2PL] model), (b) combine the interdependent items to form a polytomous item and apply a polytomous IRT model (e.g., the graded response model [GRM]), and (c) apply a model that explicitly takes into account the dependence at the item level (e.g., the testlet response theory [TRT] model). In this study, a simulation was conducted to compare the performance of these 3 approaches on number‐correct score equating when degrees of testlet effect were manipulated. The traditional equipercentile method was used as an evaluation baseline. The results show that the 2PL and the TRT approaches produce comparable results that more closely agree with the results of the equipercentile method than the GRM does. And the number‐correct equating using the 2PL is robust to the violation of local item independence.

show abstract

“…While testlet models can result in more accurate parameter estimates, there is a significant loss of information and reduction in parameters to estimate the test model (Yen, 1993;Zenisky et al, 2002). The consequence is potentially problematic for score reliability and validity of score-based decisions (Keller, Swaminathan, & Sireci, 2003).…”

Section: Testlet Modelsmentioning

confidence: 99%

Using Necessary Information to Identify Item Dependence in Passage-Based Reading Comprehension Tests

Baldonado

Svetina

Gorin

2015

Applied Measurement in Education

View full text Add to dashboard Cite

Applications of traditional unidimensional item response theory models to passage-based reading comprehension assessment data have been criticized based on potential violations of local independence. However, simple rules for determining dependency, such as including all items associated with a particular passage, may overestimate the dependency that actually exists among the items. The current study proposed a more refined method based on cognitive principles and substantive theories to determine those items that pose a threat. Specifically, the use of common necessary information from text was examined as a contributor of local dependence. Cognitively similar item pairs, those with connected necessary information, had higher local dependence values than item pairs with no connected necessary information. Results suggest that focusing on necessary information may be useful to some extent for understanding and managing item dependence for passage-based reading comprehension tests.

show abstract

Evaluating Scoring Procedures for Context-Dependent Item Sets1

Cited by 23 publications

References 20 publications

Examination of Different Item Response Theory Models on Tests Composed of Testlets

Examination of Different Item Response Theory Models on Tests Composed of Testlets

Effect of Item Response Theory (IRT) Model Selection on Testlet‐Based Test Equating

Using Necessary Information to Identify Item Dependence in Passage-Based Reading Comprehension Tests

Contact Info

Product

Resources

About