Controlling Item Exposure Conditional on Ability in Computerized Adaptive Testing

Stocking, Martha L.; Lewis, Charles

doi:10.2307/1165348

Cited by 63 publications

(100 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The item exposure parameters predicted based on the found formula were close to those observed from the Sympson and Hetter (1985) procedure and performed well in controlling item exposure rates. Similar results were observed for the Stocking and Lewis (1998) multinomial model for item selection and the Sympson and Hetter procedure with content balancing. The proposed GP approach has provided a knowledge‐based solution for finding item exposure parameters.…”

supporting

confidence: 80%

Predicting item exposure parameters in computerized adaptive testing

Chen

Doong

2008

Brit J Math & Statis

View full text Add to dashboard Cite

The purpose of this study is to find a formula that describes the relationship between item exposure parameters and item parameters in computerized adaptive tests by using genetic programming (GP) - a biologically inspired artificial intelligence technique. Based on the formula, item exposure parameters for new parallel item pools can be predicted without conducting additional iterative simulations. Results show that an interesting formula between item exposure parameters and item parameters in a pool can be found by using GP. The item exposure parameters predicted based on the found formula were close to those observed from the Sympson and Hetter (1985) procedure and performed well in controlling item exposure rates. Similar results were observed for the Stocking and Lewis (1998) multinomial model for item selection and the Sympson and Hetter procedure with content balancing. The proposed GP approach has provided a knowledge-based solution for finding item exposure parameters.

show abstract

supporting

confidence: 80%

Predicting item exposure parameters in computerized adaptive testing

Chen

Doong

2008

Brit J Math & Statis

View full text Add to dashboard Cite

show abstract

“…The exposure‐control‐only condition represents the situation where a high‐stakes test measures a single ability with homogeneous items for which no content balance is necessary but security is of major concern. Although the SH procedure has been extended to accommodate exposure control conditional on additional elements, with the Stocking and Lewis (1998) procedure deriving for each item an exposure control parameter with respect to each of a number of ability levels, and the Davey and Parshall (1995) procedure providing an exposure control parameter for each item that is conditional on all other items previously administered, the refined procedures involve much more time‐consuming iterative simulations and result in further lessened efficiency, and therefore they were not chosen for this study. Because the current study was primarily concerned with item selection methods, the general SH procedure would suffice.…”

Section: Methodsmentioning

confidence: 99%

Stratified and Maximum Information Item Selection Procedures in Computer Adaptive Testing

Deng

Ansley

Chang

2010

J Educational Measurement

View full text Add to dashboard Cite

In this study we evaluated and compared three item selection procedures: the maximum Fisher information procedure (F), the a‐stratified multistage computer adaptive testing (CAT) (STR), and a refined stratification procedure that allows more items to be selected from the high a strata and fewer items from the low a strata (USTR), along with completely random item selection (RAN). The comparisons were with respect to error variances, reliability of ability estimates and item usage through CATs simulated under nine test conditions of various practical constraints and item selection space. The results showed that F had an apparent precision advantage over STR and USTR under unconstrained item selection, but with very poor item usage. USTR reduced error variances for STR under various conditions, with small compromises in item usage. Compared to F, USTR enhanced item usage while achieving comparable precision in ability estimates; it achieved a precision level similar to F with improved item usage when items were selected under exposure control and with limited item selection space. The results provide implications for choosing an appropriate item selection procedure in applied settings.

show abstract

“…Any of the known methods for item‐exposure control can be used in combination with the three methods of item selection. Though Kingsbury and Zara recommended the McBride‐Martin method of random item selection for the spiraling method, it can be used with other exposure control methods as well, for example, the Sympson‐Hetter (1985, October) method or the conditional version thereof (Stocking & Lewis, 1998). The WDM is typically used in combination with the conditional version of the Sympson‐Hetter method.…”

Section: Comparison Between Methodsmentioning

confidence: 99%

A Comparison of Item‐Selection Methods for Adaptive Tests with Content Constraints

Linden

2005

J Educational Measurement

View full text Add to dashboard Cite

In test assembly, a fundamental difference exists between algorithms that select a test sequentially or simultaneously. Sequential assembly allows us to optimize an objective function at the examinee's ability estimate, such as the test information function in computerized adaptive testing. But it leads to the non‐trivial problem of how to realize a set of content constraints on the test—a problem more naturally solved by a simultaneous item‐selection method. Three main item‐selection methods in adaptive testing offer solutions to this dilemma. The spiraling method moves item selection across categories of items in the pool proportionally to the numbers needed from them. Item selection by the weighted‐deviations method (WDM) and the shadow test approach (STA) is based on projections of the future consequences of selecting an item. These two methods differ in that the former calculates a projection of a weighted sum of the attributes of the eventual test and the latter a projection of the test itself. The pros and cons of these methods are analyzed. An empirical comparison between the WDM and STA was conducted for an adaptive version of the Law School Admission Test (LSAT), which showed equally good item‐exposure rates but violations of some of the constraints and larger bias and inaccuracy of the ability estimator for the WDM.

show abstract

Controlling Item Exposure Conditional on Ability in Computerized Adaptive Testing

Cited by 63 publications

References 0 publications

Predicting item exposure parameters in computerized adaptive testing

Predicting item exposure parameters in computerized adaptive testing

Stratified and Maximum Information Item Selection Procedures in Computer Adaptive Testing

A Comparison of Item‐Selection Methods for Adaptive Tests with Content Constraints

Contact Info

Product

Resources

About