The Relationship Between Item Exposure and Test Overlap in Computerized Adaptive Testing

Chen, Shuying; Ankenmann, Robert D.; Spray, Judith A.

doi:10.1111/j.1745-3984.2003.tb01100.x

Cited by 55 publications

(78 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This method thus imposes the content constraints by “brute force.” It does not look at future consequences when selecting an item, but simply picks the best item from a set when its turn has arrived according to the minimax principle. CAT studies in which this method has been used have been reported in Chen, Ankenmann, and Spray (1999, April) and Leung, Chang, Hau, and Wen (2001, April; 2003, April).…”

Section: Item‐selection Methodsmentioning

confidence: 99%

A Comparison of Item‐Selection Methods for Adaptive Tests with Content Constraints

Linden

2005

J Educational Measurement

View full text Add to dashboard Cite

In test assembly, a fundamental difference exists between algorithms that select a test sequentially or simultaneously. Sequential assembly allows us to optimize an objective function at the examinee's ability estimate, such as the test information function in computerized adaptive testing. But it leads to the non‐trivial problem of how to realize a set of content constraints on the test—a problem more naturally solved by a simultaneous item‐selection method. Three main item‐selection methods in adaptive testing offer solutions to this dilemma. The spiraling method moves item selection across categories of items in the pool proportionally to the numbers needed from them. Item selection by the weighted‐deviations method (WDM) and the shadow test approach (STA) is based on projections of the future consequences of selecting an item. These two methods differ in that the former calculates a projection of a weighted sum of the attributes of the eventual test and the latter a projection of the test itself. The pros and cons of these methods are analyzed. An empirical comparison between the WDM and STA was conducted for an adaptive version of the Law School Admission Test (LSAT), which showed equally good item‐exposure rates but violations of some of the constraints and larger bias and inaccuracy of the ability estimator for the WDM.

show abstract

Section: Item‐selection Methodsmentioning

confidence: 99%

A Comparison of Item‐Selection Methods for Adaptive Tests with Content Constraints

Linden

2005

J Educational Measurement

View full text Add to dashboard Cite

show abstract

“…To control item exposure and test overlap simultaneously, the relationship between the two indices needs to be identified first. The relationship between item exposure and pairwise test overlap has been provided by Chen, Ankenmann, and Spray (2003) and can be expressed as:…”

Section: Introductionmentioning

confidence: 99%

Investigating the relationship between item exposure and test overlap: Item sharing and item pooling

Chen

Lei

2010

Brit J Math & Statis

Self Cite

View full text Add to dashboard Cite

To date, exposure control procedures that are designed to control item exposure and test overlap simultaneously are based on the assumption of item sharing between pairs of examinees. However, examinees may obtain test information from more than one examinee in practice. This larger scope of information sharing needs to be taken into account in refining exposure control procedures. To control item exposure and test overlap among a group of examinees larger than two, the relationship between the two indices needs to be identified first. The purpose of this paper is to analytically derive the relationships between item exposure rate and each of the two forms of test overlap, item sharing and item pooling, for fixed-length computerized adaptive tests. Item sharing is defined as the number of common items shared by all examinees in a group, while item pooling is the number of overlapping items that an examinee has with a group of examinees. The accuracy of the derived relationships was verified using numerical examples. The relationships derived will lay the foundation for future development of procedures to simultaneously control item exposure and item sharing or item pooling among a group of examinees larger than two.

show abstract

“…The lower the maximum item exposure rate is, the more even the usage of the item bank is, and the smaller the mean of test overlap rates is. This relation is expected and has been mathematically proved by Chen, Ankenmann, and Spray (2003).…”

Section: Test Securitymentioning

confidence: 83%

On-the-Fly Assembled Multistage Adaptive Testing

Zheng

Chang

2014

Applied Psychological Measurement

View full text Add to dashboard Cite

Recently, multistage testing (MST) has been adopted by several important large-scale testing programs and become popular among practitioners and researchers. Stemming from the decades of history of computerized adaptive testing (CAT), the rapidly growing MST alleviates several major problems of earlier CAT applications. Nevertheless, MST is only one among all possible solutions to these problems. This article presents a new adaptive testing design, “on-the-fly assembled multistage adaptive testing” (OMST), which combines the benefits of CAT and MST and offsets their limitations. Moreover, OMST also provides some unique advantages over both CAT and MST. A simulation study was conducted to compare OMST with MST and CAT, and the results demonstrated the promising features of OMST. Finally, the “Discussion” section provides suggestions on possible future adaptive testing designs based on the OMST framework, which could provide great flexibility for adaptive tests in the digital future and open an avenue for all types of hybrid designs based on the different needs of specific tests.

show abstract

The Relationship Between Item Exposure and Test Overlap in Computerized Adaptive Testing

Cited by 55 publications

References 10 publications

A Comparison of Item‐Selection Methods for Adaptive Tests with Content Constraints

A Comparison of Item‐Selection Methods for Adaptive Tests with Content Constraints

Investigating the relationship between item exposure and test overlap: Item sharing and item pooling

On-the-Fly Assembled Multistage Adaptive Testing

Contact Info

Product

Resources

About