Application of Optimal Designs to Item Calibration

Lu, Hung-Yi

doi:10.1371/journal.pone.0106747

Cited by 8 publications

(6 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Further, an opportunity in computerized calibration is to re-estimate the item parameters from the ongoing calibration and to apply a sequential optimal design, see Lu (2014), van der Linden and Ren (2015) and Ren et al (2017). This sequential and the Bayesian (or minimax) approach can also be combined.…”

Section: Discussionmentioning

confidence: 99%

“…For designing the calibration part, we will apply optimal design theory, see e.g., Atkinson, Donev and Tobias (2007). The use of optimal design theory for item calibration has been discussed previously and designs have been elaborated, see e.g., Berger (1992), Buyske (2005), Lu (2014), Zheng (2014), van der Linden and Ren (2015), Ren, van der Linden and Diao (2017).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Optimal Item Calibration for Computerized Achievement Tests

Hassan

Miller

2019

Psychometrika

View full text Add to dashboard Cite

Item calibration is a technique to estimate characteristics of questions (called items) for achievement tests. In computerized tests, item calibration is an important tool for maintaining, updating and developing new items for an item bank. To efficiently sample examinees with specific ability levels for this calibration, we use optimal design theory assuming that the probability to answer correctly follows an item response model. Locally optimal unrestricted designs have usually a few design points for ability. In practice, it is hard to sample examinees from a population with these specific ability levels due to unavailability or limited availability of examinees. To counter this problem, we use the concept of optimal restricted designs and show that this concept naturally fits to item calibration. We prove an equivalence theorem needed to verify optimality of a design. Locally optimal restricted designs provide intervals of ability levels for optimal calibration of an item. When assuming a two-parameter logistic model, several scenarios with D-optimal restricted designs are presented for calibration of a single item and simultaneous calibration of several items. These scenarios show that the naive way to sample examinees around unrestricted design points is not optimal. Electronic supplementary materialThe online version of this article (10.1007/s11336-019-09673-6) contains supplementary material, which is available to authorized users.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Optimal Item Calibration for Computerized Achievement Tests

Hassan

Miller

2019

Psychometrika

View full text Add to dashboard Cite

show abstract

“…where small values denote examinees with lower ability levels to solve the item and large values denote examinees with higher ability levels. Similar to Lu (2014), Passos and Berger (2004), and Berger, King, and Wong (2000), we assume that the experimenter is able to find examinees whose abilities match the ability levels of the optimal design. One application of the 2PL model is in computerized adaptive testing (CAT), which is a computer-based test that tailors the items to the examinee's ability.…”

Section: Bayesian D-optimal Designs For Test-item Calibrationmentioning

confidence: 99%

A Metaheuristic Adaptive Cubature Based Algorithm to Find Bayesian Optimal Designs for Nonlinear Models

Masoudi

Holling

Duarte

et al. 2019

Journal of Computational and Graphical Statistics

View full text Add to dashboard Cite

“…Item pool construction requires the estimation of the information function of the items, a step that is performed using and Item Response Theory 80 (IRT), see Veldkamp (2013) for a basic description and a critique of IRT, and Lu (2014) for a description of an estimation method. The information function is then discretized into specific ability levels of interest (such as the pass/fail ability level for an examination test) that provide coefficients for the objective and/or the constraints of the combinatorial optimization model.…”

Section: Literature Reviewmentioning

confidence: 99%

Variable neighborhood search heuristics for a test assembly design problem

Pereira

Vilà

2015

Expert Systems with Applications

View full text Add to dashboard Cite

Test assembly design problems appear in the areas of psychology and education, among others. The goal of these problems is to construct one or multiple tests to evaluate some criteria. This paper studies a recent formulation of the problem known as the one-dimensional minimax bin-packing problem with bin size constraints (MINIMAX BSC). In the MINIMAX BSC, items are initially divided into groups and multiple tests need to be constructed using a single item from each group, while minimizing the difference among the tests. We first show that the problem is NP-Hard, which remained an open question. Second, we propose three different local search neighborhoods derived from the exact resolution of special cases of the problem, and combine them into a Variable Neighborhood Search (VNS) metaheuristic. Finally, we test the proposed algorithm using real-life-based instances. The results show that the algorithm is able to obtain optimal or near-optimal solutions for instances with item pools with up to 60.000 items. Consequently, the algorithm is a viable option to design large-scale tests, as well as to provide tests for online small-sized situations such as those found in e-learning platforms.

show abstract

Application of Optimal Designs to Item Calibration

Cited by 8 publications

References 27 publications

Optimal Item Calibration for Computerized Achievement Tests

Optimal Item Calibration for Computerized Achievement Tests

A Metaheuristic Adaptive Cubature Based Algorithm to Find Bayesian Optimal Designs for Nonlinear Models

Variable neighborhood search heuristics for a test assembly design problem

Contact Info

Product

Resources

About