Optimal Design of Item Banks for Computerized Adaptive Tests

Stocking, Martha L.; Swanson, Len

doi:10.1177/01466216980223007

Cited by 26 publications

(21 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They differ in the number of parameters estimated, their estimation procedures and the number of estimation cycles, their functional form, and whether ability or item estimates are fixed during various stages of the procedure. For our study, we employed Stocking's method A [31]. This was an appropriate choice for this project since the person scores were estimated from the CAT assessments, often restricted to seven items or fewer.…”

Section: Methodsmentioning

confidence: 99%

Replenishing a computerized adaptive test of patient-reported daily activity functioning

Haley

Jette

et al. 2009

Qual Life Res

View full text Add to dashboard Cite

Purpose Computerized adaptive testing (CAT) item banks may need to be updated, but before new items can be added, they must be linked to the previous CAT. The purpose of this study was to evaluate 41 pretest items prior to including them into an operational CAT. Methods We recruited 6,882 patients with spine, lower extremity, upper extremity, and nonorthopedic impairments who received outpatient rehabilitation in one of 147 clinics across 13 states of the USA. Forty-one new Daily Activity (DA) items were administered along with the Activity Measure for Post-Acute Care Daily Activity CAT (DA-CAT-1) in five separate waves. We compared the scoring consistency with the full item bank, test information function (TIF), person standard errors (SEs), and content range of the DA-CAT-1 to the new CAT (DA-CAT-2) with the pretest items by real data simulations. Results We retained 29 of the 41 pretest items. Scores from the DA-CAT-2 were more consistent (ICC = 0.90 versus 0.96) than DA-CAT-1 when compared with the full item bank. TIF and person SEs were improved for persons with higher levels of DA functioning, and ceiling effects were reduced from 16.1% to 6.1%. Conclusions Item response theory and online calibration methods were valuable in improving the DA-CAT.

show abstract

Section: Methodsmentioning

confidence: 99%

Replenishing a computerized adaptive test of patient-reported daily activity functioning

Haley

Jette

et al. 2009

Qual Life Res

View full text Add to dashboard Cite

show abstract

“…Overlapping pools are assembled by first calculating the required numbers of pools the items should figure in and then assigning the items to the pool9 until these numbers are realiLed. For empirical results with these methods, see Stocking and Swanson (1998).…”

Section: Current Methods To Construct Rotating Item Poolsmentioning

confidence: 99%

“…Veldkamp and van der Linden (2000) suggested to calculate a blueprint for the item pool such that distribution of the exposure rates for the items in the pool tends to be even. Stocking and Swanson (1998) introduced a method of rotating item pools with the same goal of even exposure rates. The item pools in this method are assembled from a master pool by splitting it into several smaller pools.…”

mentioning

confidence: 99%

Constructing Rotating Item Pools for Constrained Adaptive Testing

Ariel

Veldkamp

Linden

2004

J Educational Measurement

View full text Add to dashboard Cite

Preventing items in adaptive testing from being over-or underexposed is one of the main problems in computerized adaptive testing. Though the problem of overexposed items can be solved using a probabilistic item-exposure control method, such methods are unable to deal with the problem of underexposed items. Using a system of rotating item pools, on the other hand, is a method that potentially solves both problems. In this method, a master pool is divided into (possibly overlapping) smaller item pools that are required to have similar distributions of content and statistical attributes. These pools are rotated among the testing sites to realize desirable exposure rates for the items. In this paper, a test assembly model for the problem of dividing a master pool into a set of smaller pools is presented. The model was motivated by Gullicksen's (1950) matched random subtests method. Different methods to solve the model are proposed. An item pool from the Law School Admission Test was used to evaluate the performances of computerized adaptive tests from systems of rotating item pools constructed using these methods. (Contains 6 figures and 14 references.) (Author/SLD)Reproductions supplied by EDRS are the best that can be made from the original document. AbstractPreventing items in adaptive testing from being over-or underexposed is one of the main problems in computerized adaptive testing. Though the problem of overexposed items can be solved using a probabilistic item-exposure control method, such methods are unable to deal with the problem of underexposed items. Using a system of rotating item pools, on the other hand, is a method that potentially solves both problems. In this method, a master pool is divided into (possibly overlapping) smaller item pools which are required to have similar distributions of content and statistical attributes. These pools are rotated among the testing sites to realize desirable exposure rates for the items. In this paper, a test assembly model for the problem of dividing a master pool into a set of smaller pools is presented. The model was motivated by Gulliksen's (1950) matched random subtests method. Different methods to solve the model are proposed. An item pool from the Law School Admission Test (LSAT) was used to evaluate the performances of computerized adaptive test from systems of rotating item pools constructed using these methods.

show abstract

“…Both Swanson and Stocking (1998) and Way, Steffen and Anderson (1998;see also Way,1998) address the problem of designing a system of rotating item pools for CAT.…”

Section: Item Pool Designmentioning

confidence: 99%

Designing Item Pools for Computerized Adaptive Testing

Veldkamp

Linden

2000

Computerized Adaptive Testing: Theory and Practice

View full text Add to dashboard Cite

A method of item pool design is proposed that uses an optimal blueprint for the item pool calculated from the test specifications. The blueprint is a document that specifies the attributes that the items in the computerized adaptive test (CAT) pool should have. The blueprint can be a starting point for the item writing process, and it can be used to assemble item pools in a system of rotating pools from a master pool. The blueprint is also useful for item pool maintenance. Designing the blueprint begins with analyzing the specifications for the CAT, a step amounting to the formation of a classification table involving categorization of quantitative item attributes. Using this table, an integer programming model for the assembly of the shadow tests in the CAT simulation is constructed. An estimate of the ability distribution of the identified population of examinees is obtained, and the CAT simulation is carried out using the integer programming model for the shadow tests and sampling simulees from the ability distribution. The blueprint is then calculated from the counts of the number of items from the cells in the classification table. The best way to implement the blueprint is in a sequential fashion recalculating the blueprint after a certain portion of the items has actually been written and tested so that their attribute values are known. (Contains 16 references and a list

show abstract

Optimal Design of Item Banks for Computerized Adaptive Tests

Cited by 26 publications

References 9 publications

Replenishing a computerized adaptive test of patient-reported daily activity functioning

Replenishing a computerized adaptive test of patient-reported daily activity functioning

Constructing Rotating Item Pools for Constrained Adaptive Testing

Designing Item Pools for Computerized Adaptive Testing

Contact Info

Product

Resources

About