A Comparison of Item‐Selection Methods for Adaptive Tests with Content Constraints

Linden, Willem J. van der

doi:10.1111/j.1745-3984.2005.00015.x

Cited by 33 publications

(29 citation statements)

References 21 publications

(46 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In a separate study, the WDA and STA were compared using the content specifications of an existing large-scale testing program not run by ETS and an item pool developed independently of either method. For a report on this study, which resulted in different findings, see van der Linden (2003).…”

Section: Evaluation Methodsmentioning

confidence: 94%

A Comparison of Two Procedures for Constrained Adaptive Test Construction

Robin

Linden

Eignor

et al. 2004

ETS Research Report Series

Self Cite

View full text Add to dashboard Cite

The relatively new shadow test approach (STA) to computerized adaptive testing (CAT) proposed by Wim van der Linden is a potentially attractive alternative to the weighted deviation algorithm (WDA) implemented at ETS. However, it has not been evaluated under testing conditions representative of current ETS testing programs. Of interest was whether STA would, under typical high‐stakes on‐demand testing situations, produce tests of comparable or better psychometric quality as those produced by the current weighted deviation algorithm. Based on simulated data, we found that the STA performed as well or slightly better than the WDA on two of the three commonly accepted testing objectives: measurement and content. The WDA appeared to perform slightly better than the STA when the issue is security or item exposure control. The paper provides a review of the rationale that led to the specific testing objectives employed, an outline of the test construction steps common to the two procedures investigated, and a description of the specific models and algorithms employed by both procedures. Detailed description of the simulation study conducted and the results obtained from both of the procedures for one of the pools are also provided. Results are summarized and further research needs in particular areas discussed.

show abstract

Section: Evaluation Methodsmentioning

confidence: 94%

A Comparison of Two Procedures for Constrained Adaptive Test Construction

Robin

Linden

Eignor

et al. 2004

ETS Research Report Series

Self Cite

View full text Add to dashboard Cite

show abstract

“…Several studies have compared the performance of various approaches used to accommodate LID in CAT settings, against one another and / or against a traditional approach in which LID concerns are ignored (Boyd, 2003;Keng, 2008;Schnipke & Reese, 1997;Stocking & Swanson, 1993;van der Linden, 2005;van der Linden & Reese, 1998). Results generally show that although the alternative approaches are slightly less efficient in terms of measurement precision, the differences are often of practical negligence.…”

Section: Prevent Lid Through Test Constructionmentioning

confidence: 99%

Detecting Local Item Dependence in Polytomous Adaptive Data

Mislevy

Rupp

Harring

2012

J Educational Measurement

View full text Add to dashboard Cite

iii Table 6. Results summary for tracked item pairs in the CONV condition with no LID .. 91 1 For simplicity, the term "individual" will be used throughout the document to denote the person who provides the responses to a data-collection instrument; the term is assumed to be synonymous with alternative terms such as "examinee" and "respondent," though it is acknowledged that each has domainspecific connotations. Along these lines, the data-collection instrument will be referred to as an "instrument," (representing alternative terms such as "test" and "questionnaire") and the term "trait" is used to denote the construct measured by the items (representing alternative terms such as "ability" and "proficiency"). Exceptions are made in rare instances when necessary to remain consistent with the literature. 2This initiative focuses on more accurate and efficient measurement of patient-reported symptoms and aspects of health-related quality of life. A primary goal of PROMIS is to develop instruments based on IRT methodologies in these domains that are publically available for the clinical research community.In IRT, estimates of respondents' traits (θ) are based not only on the responses they provide, but also the characteristics (i.e., parameters) of the items they are administered such as their difficulty -reflected by category boundary parameters (b) -their ability to differentiate among respondents -reflected by slope parameters (a) -and their susceptibility to guessing -reflected by lower asymptote parameters (c).One unidimensional IRT model frequently applied in health-outcomes settings is the graded response model (GRM;Samejima, 1969). The GRM is appropriate for item responses that fall in multiple ordered categories. It predicts the conditional probability of an individual responding in a particular category as a function of an individual's latent trait value and several item parameters.The GRM is considered a "difference model" (Thissen & Steinberg, 1986) or "indirect IRT model" (Embretson & Reise, 2000) because the probabilities are computed in two stages. Following Dodd, De Ayala, and Koch (1995), the probability that individual i will produce a response in category k or higher for item j is first computed as:where θ i is the individual's trait level, a j is the discrimination parameter for item j, and b jk is the category boundary for category k for item j. In the second step, the actual category 3 response probabilities for each response are computed by subtracting the cumulative probabilities from adjacent response categories conditional on θ using:There are a total of K -1 boundary parameters that need to be estimated for an item with K score categories. Figure 1 illustrates the probability of choosing each of the response options offered with an item from PROMIS designed to measure fatigue impact and experience according to the GRM. It shows that a person with a low level of fatigue would have a high probability of indicating that his fatigue made it "not at all hard to carry on a conversation." In ...

show abstract

“…Az adaptív tesztek számos változata létezik (van der Linden, 2008), az itemalapú tesztektől a lineáris alteszteket alkalmazó többszakaszos tesztekig, alapelvüket tekintve azonban mindegyik adaptív teszt hasonlóan épül fel. A fejezet célja, hogy áttekintést adjunk Az adaptív tesztek működése szigorú algoritmushoz kötött (Linacre, 2000).…”

Section: I6 Az Adaptív Tesztkészítés Folyamataunclassified

“…Egyrészt alapfeltétel a több száz itemből álló kalibrált feladatbank létrehozása, melynek kifejlesztése szakembereket igényel, másrészt a számítógép alapú kiközvetítéshez speciális szoftverekre van szükség, melyek szintén szakembereket és jelentős anyagi forrásokat kívánnak (Thompson és Weiss, 2011;van der Linden, 2008).…”

Section: I61 Megvalósíthatóság Alkalmazhatóság Lehetőségeinek Felmunclassified

“…Az item kiválasztási gyakoriságának szabályozása során az algoritmus azt vezérli, hogy nehogy bizonyos itemek túl sokszor szerepeljenek, ami veszélyeztetné a tesztbiztonságot. A tartalmi szabályozás a különböző tartalmi elemek azonos mértékű kiválasztását kontrollálja (van der Linden, 2005). Az egymásnak információt szolgáltató itemeken alkalmazott szabályozás kiküszöböli a nagyon hasonló, vagy egymásnak információt szolgáltatható elemek kiválasztását (Weiss, 2013 Az egyik legelterjedtebb módszer a legnagyobb valószínűség módszere, melynél mindig a legmagasabb információtartalommal rendelkező item választódik ki (Kingsbury és Zara, 1989;Weiss, 2013;Schnipke és Green, 1995).…”

Section: I66 Itemkiválasztási Algoritmus Meghatározásaunclassified

See 1 more Smart Citation

Számítógép alapú adaptív és lineáris tesztek összehasonlító hatékonyságvizsgálata

Andrea¹

View full text Add to dashboard Cite

A Comparison of Item‐Selection Methods for Adaptive Tests with Content Constraints

Cited by 33 publications

References 21 publications

A Comparison of Two Procedures for Constrained Adaptive Test Construction

A Comparison of Two Procedures for Constrained Adaptive Test Construction

Detecting Local Item Dependence in Polytomous Adaptive Data

Számítógép alapú adaptív és lineáris tesztek összehasonlító hatékonyságvizsgálata

Contact Info

Product

Resources

About