A Comparison of Equating Methods and Linking Designs for Developing an Item Pool under Item Response Theory

Arai, Sayaka; Mayekawa, Shin-ichi

doi:10.2333/bhmk.38.1

Cited by 26 publications

(34 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This pattern held in all conditions with a latent proficiency shift, albeit eliminating items with strong DIF diminished the bias. A similar bias in estimation using FCIP when a latent proficiency shift was present has been reported before (Arai & Mayekawa, ; Baldwin et al., ; Hu et al., ; Kang & Petersen, ; Keller & Keller, ; Kim, ; Paek & Young, ). Our simulation showed that this effect did not occur in conditions without DIF or with negligible DIF.…”

Section: Discussionsupporting

confidence: 81%

“…Further, some research has indicated that the performance of FCIP is different from transformation methods such as mean‐mean linking (MM; Loyd & Hoover, ), mean‐sigma linking (Marco, ), or characteristic curve linking (Haebara, ; Stocking & Lord, ) and may yield biased results when there is a shift in the population that leads to changes in the latent ability distribution (Baldwin, Baldwin, & Nering, ; Hu, Rogers, & Vukmirovic, ; Keller & Keller, ; Kim, ; Paek & Young, ). This effect occurs most often when only a small number of common items are employed (Arai & Mayekawa, ) and it depends on the software implementation (Kang & Petersen, ; Kim, ; Paek & Young, ). According to Stocking and Lord (SL; ), an alternative to the FCIP procedure is characteristic curve linking.…”

mentioning

confidence: 99%

See 1 more Smart Citation

A Comparison of Linking Methods for Estimating National Trends in International Comparative Large‐Scale Assessments in the Presence of Cross‐National DIF

Sachse

Roppelt

Haag

2016

J Educational Measurement

View full text Add to dashboard Cite

Trend estimation in international comparative large-scale assessments relies on measurement invariance between countries. However, cross-national differential item functioning (DIF) has been repeatedly documented. We ran a simulation study using national item parameters, which required trends to be computed separately for each country, to compare trend estimation performances to two linking methods employing international item parameters across several conditions. The trend estimates based on the national item parameters were more accurate than the trend estimates based on the international item parameters when cross-national DIF was present. Moreover, the use of fixed common item parameter calibrations led to biased trend estimates. The detection and elimination of DIF can reduce this bias but is also likely to increase the total error. 152

show abstract

Section: Discussionsupporting

confidence: 81%

mentioning

confidence: 99%

A Comparison of Linking Methods for Estimating National Trends in International Comparative Large‐Scale Assessments in the Presence of Cross‐National DIF

Sachse

Roppelt

Haag

2016

J Educational Measurement

View full text Add to dashboard Cite

show abstract

“…This study evaluates MFRM-based performance-test linking accuracy through the following simulation procedure, which is based on a typical experimental method for evaluating IRT-based objective test linking accuracy (Lee & Ban, 2009;Arai & Mayekawa, 2011;Kilmen & Demirtasli, 2012;Uysal & Ibrahim, 2016…”

Section: Linking Accuracy Criteriamentioning

confidence: 99%

Accuracy of performance-test linking based on a many-facet Rasch model

Uto

2020

Behav Res

View full text Add to dashboard Cite

Performance assessments, in which human raters assess examinee performance in practical tasks, have attracted much attention in various assessment contexts involving measurement of higher-order abilities. However, difficulty persists in that ability measurement accuracy strongly depends on rater and task characteristics such as rater severity and task difficulty. To resolve this problem, various item response theory (IRT) models incorporating rater and task parameters, including many-facet Rasch models (MFRMs), have been proposed. When applying such IRT models to datasets comprising results of multiple performance tests administered to different examinees, test linking is needed to unify the scale for model parameters estimated from individual test results. In test linking, test administrators generally need to design multiple tests such that raters and tasks partially overlap. The accuracy of linking under this design is highly reliant on the numbers of common raters and tasks. However, the numbers of common raters and tasks required to ensure high accuracy in test linking remain unclear, making it difficult to determine appropriate test designs. We therefore empirically evaluate the accuracy of IRT-based performance-test linking under common rater and task designs. Concretely, we conduct evaluations through simulation experiments that examine linking accuracy based on a MFRM while changing numbers of common raters and tasks with various factors that possibly affect linking accuracy.

show abstract

“…) IRT (Kolen and Brennan, 2004) concurrent calibration separate calibration (Hanson and Béguin, 2002) 0-1 IRT (multiple group IRT; Bock and Zimowski, 1996) IRT 0-1 IRT (1) 0-1 (2) Muraki, Mislevy, and Bock, 2003) (2007) Arai and Mayekawa, 2011;, 2009BILOG-MG Mislevy (1984 IRT E-M …”

Section: Item Response Theory; Irt Equatingmentioning

confidence: 99%

“…0.20 N(0.00, 1.00) 10 0-1 1 log(δ) 0.20 N(0.00, 1.00) Arai and Mayekawa (2011) ICC item characteristic curve …”

Section: Item Response Theory; Irt Equatingmentioning

confidence: 99%

An Evaluation of the Effect of Simplifying Group Structure when Multiple Group IRT Models were Used

Mitsunaga¹,

Mayekawa²

2013

Kodo Keiryogaku

Self Cite

View full text Add to dashboard Cite

Consider the situation when a set of new items are administered as the part of several forms including anchor items. In this case, in order to add the new items to the existing item bank, we could either 1) estimate the item parameters of the items in each form and equate them using the information contained in the anchor items (separate calibration), or 2) combine the new item response data to the old data set and calibrate whole the items again using the multiple group IRT (concurrent calibration). However, as the number of groups becomes large, the existing multiple group IRT calibration program fails to converge due to the lack of available memory. In this study we examined the proper method to merge groups in order to avoid the above problem. Simulation studies showed that the separate calibration estimates were closest to the true value, and the concurrent calibration with reduced groups was similar to the separate calibration if the mean values of latent ability were similar among the merged groups.

show abstract

A Comparison of Equating Methods and Linking Designs for Developing an Item Pool under Item Response Theory

Cited by 26 publications

References 13 publications

A Comparison of Linking Methods for Estimating National Trends in International Comparative Large‐Scale Assessments in the Presence of Cross‐National DIF

A Comparison of Linking Methods for Estimating National Trends in International Comparative Large‐Scale Assessments in the Presence of Cross‐National DIF

Accuracy of performance-test linking based on a many-facet Rasch model

An Evaluation of the Effect of Simplifying Group Structure when Multiple Group IRT Models were Used

Contact Info

Product

Resources

About