Online Calibration Via Variable Length Computerized Adaptive Testing

Chang, Yuan‐chin Ivan; Lu, Hung-Yi

doi:10.1007/s11336-009-9133-0

Cited by 23 publications

(35 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…A monitoring procedure can be applied in active mode if the CAT system allows the procedure to interact with the item selection algorithm (e.g., stop administering flagged items temporarily or permanently) and/or change the item pool (e.g., replace flagged items with new items of similar characteristics) during CAT administration; otherwise, the procedure can only be used in passive mode to monitor items and provide statistically significant results without interrupting the CAT item selection algorithm and without changing the item pool. Nowadays, online calibration methods are becoming more popular and are widely used to calibrate new items (Ban, Hanson, Wang, Yi, & Harris, ; Chang & Lu, ; Jones & Jin, ; Stocking, ; van der Linden & Ren, ; Wainer & Mislevy, ). Newly calibrated items together with existing new items make it possible for a procedure to monitor items in an active mode and replace flagged items with new items.…”

Section: Sequential Procedures For Monitoring Itemsmentioning

confidence: 99%

Monitoring Items in Real Time to Enhance CAT Security

Zhang

2016

J Educational Measurement

View full text Add to dashboard Cite

An IRT-based sequential procedure is developed to monitor items for enhancing test security. The procedure uses a series of statistical hypothesis tests to examine whether the statistical characteristics of each item under inspection have changed significantly during CAT administration. This procedure is compared with a previously developed CTT-based procedure through simulation studies. The results show that when the total number of examinees is fixed both procedures can control the rate of type I errors at any reasonable significance level by choosing an appropriate cutoff point and meanwhile maintain a low rate of type II errors. Further, the IRT-based method has a much lower type II error rate or more power than the CTTbased method when the number of compromised items is small (e.g., 5), which can be achieved if the IRT-based procedure can be applied in an active mode in the sense that flagged items can be replaced with new items.

show abstract

Section: Sequential Procedures For Monitoring Itemsmentioning

confidence: 99%

Monitoring Items in Real Time to Enhance CAT Security

Zhang

2016

J Educational Measurement

View full text Add to dashboard Cite

show abstract

“…The new items should be precisely calibrated before being put into the item pool for use formally. Moreover, the calibration accuracy of the new items has great influence on the estimation accuracy of the examinees' latent trait in the ensuing CAT sessions (e.g., van der Linden and Glas, 2000; Chang and Lu, 2010). Wainer and Mislevy (1990) proposed two strategies for calibrating new items based on CAT in the literature.…”

Section: Introductionmentioning

confidence: 99%

Online Calibration of Polytomous Items Under the Graded Response Model

et al. 2020

View full text Add to dashboard Cite

Computerized adaptive testing (CAT) is an efficient testing mode, which allows each examinee to answer appropriate items according his or her latent trait level. The implementation of CAT requires a large-scale item pool, and item pool needs to be frequently replenished with new items to ensure test validity and security. Online calibration is a technique to calibrate the parameters of new items in CAT, which seeds new items in the process of answering operational items, and estimates the parameters of new items through the response data of examinees on new items. The most popular estimation methods include one EM cycle method (OEM) and multiple EM cycle method (MEM) under dichotomous item response theory models. This paper extends OEM and MEM to the graded response model (GRM), a popular model for polytomous data with ordered categories. Two simulation studies were carried out to explore online calibration under a variety of conditions, including calibration design, initial item parameter calculation methods, calibration methods, calibration sample size and the number of categories. Results show that the calibration accuracy of new items were acceptable, and which were affected by the interaction of some factors, therefore some conclusions were given.

show abstract

“…The maintenance and management of item bank is essential for CAT administration, and item replenishment is even more important than P&P administration due to the accelerated exhausting of items in CAT (Y. C. I. Chang & Lu, 2010). After a period of time, some operational items may no longer be suitable for use due to reasons such as overexposure, obsoleteness, or flaw (Wainer & Mislevy, 1990), thus it is very necessary to develop new items to substitute the unsuitable ones.…”

Section: Introductionmentioning

confidence: 99%

A New Online Calibration Method Based on Lord’s Bias-Correction

Chen

et al. 2017

Applied Psychological Measurement

View full text Add to dashboard Cite

Online calibration technique has been widely employed to calibrate new items due to its advantages. Method A is the simplest online calibration method and has attracted many attentions from researchers recently. However, a key assumption of Method A is that it treats personparameter estimatesûs (obtained by maximum likelihood estimation [MLE]) as their true values us, thus the deviation of the estimatedûs from their true values might yield inaccurate item calibration when the deviation is nonignorable. To improve the performance of Method A, a new method, MLE-LBCI-Method A, is proposed. This new method combines a modified Lord's biascorrection method (named as maximum likelihood estimation-Lord's bias-correction with iteration [MLE-LBCI]) with the original Method A in an effort to correct the deviation ofûs which may adversely affect the item calibration precision. Two simulation studies were carried out to explore the performance of both MLE-LBCI and MLE-LBCI-Method A under several scenarios. Simulation results showed that MLE-LBCI could make a significant improvement over the ML ability estimates, and MLE-LBCI-Method A did outperform Method A in almost all experimental conditions.

show abstract

Online Calibration Via Variable Length Computerized Adaptive Testing

Abstract: adaptive testing, item response theory, logistic regression, measurement error, nonlinear design, online calibration, stopping rule,

Cited by 23 publications

References 39 publications

Monitoring Items in Real Time to Enhance CAT Security

Monitoring Items in Real Time to Enhance CAT Security

Online Calibration of Polytomous Items Under the Graded Response Model

A New Online Calibration Method Based on Lord’s Bias-Correction

Contact Info

Product

Resources

About