A Method for Severely Constrained Item Selection in Adaptive Testing

Stocking, Martha L.; Swanson, Len

doi:10.1002/j.2333-8504.1992.tb01468.x

Cited by 53 publications

(72 citation statements)

References 14 publications

(9 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Besides the disadvantages mentioned above, the CCAT procedure may also be inappropriate when the number of item features considered is large, in which case the number of items in each partition can be very small (Stocking & Swanson, 1993). A weighted deviation model (WDM) proposed by Stoclung & Swanson (1 993) may solve this problem in a robust way.…”

Section: Completely Randomized Item Selection (Rn)mentioning

confidence: 99%

Effects of Practical Constraints on Item Selection Rules at the Early Stages of Computerized Adaptive Testing

Chen

Ankenman

2004

J Educational Measurement

View full text Add to dashboard Cite

The purpose of this study was to compare the effects of four item selection rules-( 1 ) Fisher information (F), (2) Fisher informution with a posterior distribution (FP), (3) Kullhack-Leibler information with a posterior distribution (KP), and (4) cotnpletely randomized item selection (RN)-nith respect to the precision of trait estimation and the extent of item usage at the early stages of computerized adaptive testing. The comparison ofthe four item selection rules was carried out under three conditions: (1) using only the item informatiorz function as the item selection criterion; (2) using both the item information function and content balancing; and (3) using the item information function, content balancing, and item exposure control. When test length was less than 10 items, FP and KP tended to outper$orm F at extreme trait levels in Condition 1. However, in more realistic settings, it could not he concluded that FP and KP outperformed F, especially when item exposure control was imposed. When test length was greater than 10 items, the three nonrandom item selection procedures performed similarly no matter what the condition was, while F had slightly higher item usage.

show abstract

Section: Completely Randomized Item Selection (Rn)mentioning

confidence: 99%

Effects of Practical Constraints on Item Selection Rules at the Early Stages of Computerized Adaptive Testing

Chen

Ankenman

2004

J Educational Measurement

View full text Add to dashboard Cite

show abstract

“…3A recent trend in CAT development is to constrain item selection further to balance content (see, e.g., Stocking & Swanson, 1993) and control item exposure (see, e.g., Stocking & Lewis, 1995;Sympson & Hetter, 1985). Because it is difficult to model these constraints in a generalizable way, we did not include them in this study.…”

Section: Notesmentioning

confidence: 99%

Properties of Ability Estimation Methods in Computerized Adaptive Testing

Wang¹,

Vispoel²

1998

J Educational Measurement

View full text Add to dashboard Cite

Simulations of computerized adaptive tests (CATs) were used to evaluate results yielded by four commonly used ability estimation methods: maximum likelihood estimation (MLE) and three Bayesian approaches----Owen's method, expected a posteriori (EAP), and maximum a posteriori. In line with the theoretical nature of the ability estimates and previous empirical research, the results showed clear distinctions between MLE and the Bayesian methods, with MLE yielding lower bias, higher standard errors, higher root mean square errors, lower fidelity, and lower administrative efficiency. Standard errors for MLE based on test information underestimated actual standard errors, whereas standard errors for the Bayesian methods based on posterior distribution standard deviations accurately estimated actual standard errors. Among the Bayesian methods, Owen's provided the worst overall results, and EAP provided the best. Using a variable starting rule in which examinees were initially classified into three broad/ability groups greatly reduced the bias for the Bayesian methods, but had little effect on the results for MLE. On the basis of these results, guidelines are offered for selecting appropriate CAT ability estimation methods in different decision contexts.

show abstract

“…. lCertain of the material contained in this section draws heavily on material appearing in a paper by Stocking and Swanson (1992).…”

Section: Constraints On Intrinsic Item Propertiesmentioning

confidence: 99%

Case Studies in Computer Adaptive Test Design Through Simulation1,2

Eignor

Stocking

Way

et al. 1993

ETS Research Report Series

Self Cite

View full text Add to dashboard Cite

The extensive computer simulation work done in developing the computer adaptive versions of the Graduate Record Examinations (GRE) Board General Test and the College Board Admissions Testing Program (ATP) SAT is described in this report. Both the GRE General and SAT computer adaptive tests (CATs), which are fixed length in nature, were developed from pools of items that were calibrated using the three‐parameter logistic IRT model and item selection was based on the recently developed weighted deviations algorithm (see Swanson and Stocking, 1992), which simultaneously deals with content, statistical, and other constraints in the item selection process. For the GRE General CATs (Verbal, Quantitative, and Analytical), item exposure was controlled by using an extension of an approach originally developed by Sympson and Hetter (1988). For the SAT CATs (Verbal and Mathematical), item exposure was controlled by using a less complex randomization approach. Lengths of the CATs were determined so that CAT reliabilities matched or exceeded comparable full length paper‐and‐pencil test reliabilities.

show abstract

A Method for Severely Constrained Item Selection in Adaptive Testing

Cited by 53 publications

References 14 publications

Effects of Practical Constraints on Item Selection Rules at the Early Stages of Computerized Adaptive Testing

Effects of Practical Constraints on Item Selection Rules at the Early Stages of Computerized Adaptive Testing

Properties of Ability Estimation Methods in Computerized Adaptive Testing

Case Studies in Computer Adaptive Test Design Through Simulation1,2

Contact Info

Product

Resources

About