<i>a</i>‐Stratified CAT design with content blocking

Yi, Qing; Chang, Hua Hua

doi:10.1348/000711003770480084

Cited by 36 publications

(47 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The a-stratified methods (Chang & Ying, 1999;Chang et al, 2001;Yi & Chang, 2003) select items from a stratified pool based on the closeness between item difficulty and the current CAT ability estimate. In the a-stratified method with content blocking (STRC; Yi & Chang, 2003), an item pool is first divided into groups based on the content specifications of the pool.…”

Section: Cat Methods Investigatedmentioning

confidence: 99%

“…In the a-stratified method with content blocking (STRC; Yi & Chang, 2003), an item pool is first divided into groups based on the content specifications of the pool.…”

Section: Cat Methods Investigatedmentioning

confidence: 99%

“…For each simulee, a fixed length CAT of 40 items was simulated. A content control procedure that uses a modified multinomial model as described in Yi and Chang (2003) was implemented as part of the CAT methods so that each simulated CAT consists of about 40% items of content area one and 30% items from each of the other two content areas.…”

Section: Cat Simulationsmentioning

confidence: 99%

See 2 more Smart Citations

Severity of Organized Item Theft in Computerized Adaptive Testing: An Empirical Study

Yi¹,

Zhang²,

Chang

2006

ETS Research Report Series

Self Cite

View full text Add to dashboard Cite

Chang and Zhang (2002, 2003) proposed several baseline criteria for assessing the severity of possible test security violations for computerized tests with high-stakes outcomes. However, these criteria were obtained from theoretical derivations that assumed uniformly randomized item selection. The current study investigated potential damage caused by organized item theft in computerized adaptive testing (CAT) for two more realistic item selection methods, the maximum item information and the a-stratified, while using the randomized method as a baseline for comparison. The results of the study indicated that the damage could be very severe, especially when the thieves took the test in the early stage of utilization of an item pool. Among the three CAT methods examined in this study, the maximum item information method with Sympson-Hetter exposure control was most vulnerable to organized item theft.

show abstract

Section: Cat Methods Investigatedmentioning

confidence: 99%

“…In the a-stratified method with content blocking (STRC; Yi & Chang, 2003), an item pool is first divided into groups based on the content specifications of the pool.…”

Section: Cat Methods Investigatedmentioning

confidence: 99%

Section: Cat Simulationsmentioning

confidence: 99%

See 1 more Smart Citation

Severity of Organized Item Theft in Computerized Adaptive Testing: An Empirical Study

Yi¹,

Zhang²,

Chang

2006

ETS Research Report Series

Self Cite

View full text Add to dashboard Cite

show abstract

“…Different item selection methods have different degrees of resistance to test security breaches. For any specific item selection method, such as the weighted deviation method , the a-stratified method (Chang & Ying, 1999;Yi & Chang, 2003), and constrained CAT with shadow tests (van der Linden, 2000), a comparison of test security between the single-pool approach and the multiple-pool approach can be conducted by a simulation study. Below, we discuss a simulation study using a popular CAT item selection method, the maximum item information method (Lord, 1980) with the Sympson-Hetter item exposure control (Hetter & Sympson, 1997;Sympson & Hetter, 1985), to find out whether a two-pool design outperforms a single-pool design in terms of test security.…”

Section: Abstract Computerized Testing Adaptive Testing Item Respmentioning

confidence: 99%

“…A content control procedure based on a modified multinomial model (Yi & Chang, 2003) was implemented such that each simulated test consisted of about 40% of the items from Content Area 1 and 30% of the items from each of the other two content areas.…”

Section: Simulation Studymentioning

confidence: 99%

Comparing single-pool and multiple-pool designs regarding test security in computerized testing

Zhang

Chang

2012

Behav Res

Self Cite

View full text Add to dashboard Cite

This article compares the use of single-and multiple-item pools with respect to test security against item sharing among some examinees in computerized testing. A simulation study was conducted to make a comparison among different pool designs using the item selection method of maximum item information with the Sympson-Hetter exposure control and content balance. The results from the simulation study indicate that two-pool designs have a better degree of resistance to item sharing than do the single-pool design in terms of measurement precision in ability estimation. This article further characterizes the conditions under which employing a multiple-pool design is better than using a single, whole pool in terms of minimizing the number of compromised items encountered by examinees under a randomized item selection method. Although no current computerized testing program endorses the randomized item selection method, the results derived in this study can shed some light on item pool designs regarding test security for all item selection algorithms, especially those that try to equalize or balance item exposure rates by employing a randomized item selection method locally, such as the astratified-with-b-blocking method.

show abstract

Item Exposure

Robin¹

2005

Encyclopedia of Statistics in Behavioral Science

View full text Add to dashboard Cite

Often necessary in practice, the reuse of items over administrations is a threat to the validity of test scores. This entry briefly describes the nature of that threat and shows how item exposure can be used as a means for continuously evaluating, managing and monitoring it. This entry also outlines major strategies and methodologies for controlling item exposure within levels that testing programs can find acceptable.

show abstract

a‐Stratified CAT design with content blocking

Cited by 36 publications

References 25 publications

Severity of Organized Item Theft in Computerized Adaptive Testing: An Empirical Study

Severity of Organized Item Theft in Computerized Adaptive Testing: An Empirical Study

Comparing single-pool and multiple-pool designs regarding test security in computerized testing

Item Exposure

Contact Info

Product

Resources

About