Operational Characteristics of Adaptive Testing Procedures Using the Graded Response Model

Dodd, Barbara G.; Koch, William R.; Ayala, R. J. De

doi:10.1177/014662168901300202

Cited by 57 publications

(76 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To overcome this problem, in most cases a systematic procedure to estimate 8 is used, until item scores in two different item categories are observed, and after this, maximum likelihood is used to estimate 0 (see e.g., De Ayala, 1989 Dodd et al (1989).…”

Section: Polytomous Item Response Modelsmentioning

confidence: 99%

Detection of Person Misfit in Computerized Adaptive Tests with Polytomous Items

Krimpen-Stoop¹,

Meijer

2002

Applied Psychological Measurement

View full text Add to dashboard Cite

Item scores that do not fit an assumed item response theory model may cause the latent trait value to be estimated inaccurately. For computerized adaptive tests (CAT) with dichotomous items, several person-fit statistics for detecting nonfitting item score patterns have been proposed. Both for paper-and-pencil (P&P) test and CATs, detection of person misfit with polytomous items has hardly been explored. In this simulation study, the theoretical and empirical null distributions of a person-fit statistic for polytomous items are compared for P&P tests and CATs. Results show that the empirical distribution of this statistic was close to the standard normal distribution, for both P&P tests and CATs. Also statistics that are especially designed for a CAT are proposed. In these statistics observed and expected item scales are compared using cumulative sum (CUSUM) procedures.

show abstract

Section: Polytomous Item Response Modelsmentioning

confidence: 99%

Detection of Person Misfit in Computerized Adaptive Tests with Polytomous Items

Krimpen-Stoop¹,

Meijer

2002

Applied Psychological Measurement

View full text Add to dashboard Cite

show abstract

Section: How Cat Operatesmentioning

confidence: 99%

“…The most critical distinction, however, is that polytomously scored items are more informative than dichotomously scored items, which has implications for the size of the item bank and test length Dodd, Koch, & De Ayala, 1989Singh, Howell, & Rhoads, 1990;Thissen et al, 2007).…”

Section: How Cat Operatesmentioning

confidence: 99%

“…Banks of polytomously scored items can be smaller, and a CAT can be implemented successfully on banks containing as few as 30 or 40 items that are sufficiently spread along the latent continuum (Craig & Harvey, 2004;Dodd, Koch, & De Ayala, 1989Singh, Howell, & Rhoads, 1990). Second, assuming the stopping rule is based on the standard error of the trait estimate, adaptive instruments containing polytomously scored items can be shorter than those containing dichotomously scored items.…”

Section: How Cat Operatesmentioning

confidence: 99%

“…First, item banks consisting of polytomously scored items can be smaller (i.e., contain fewer items) than banks consisting of dichotomously scored items (Dodd, Koch, & De Ayala, 1989. To implement a CAT based on dichotomously scored items, Urry (1977) recommends that the bank contain at least 100 items (ignoring other test specifications such as exposure controls and content balancing).Banks of polytomously scored items can be smaller, and a CAT can be implemented successfully on banks containing as few as 30 or 40 items that are sufficiently spread along the latent continuum (Craig & Harvey, 2004;Dodd, Koch, & De Ayala, 1989Singh, Howell, & Rhoads, 1990). Second, assuming the stopping rule is based on the standard error of the trait estimate, adaptive instruments containing polytomously scored items can be shorter than those containing dichotomously scored items.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Detecting Local Item Dependence in Polytomous Adaptive Data

Mislevy

Rupp

Harring

2012

J Educational Measurement

View full text Add to dashboard Cite

iii Table 6. Results summary for tracked item pairs in the CONV condition with no LID .. 91 1 For simplicity, the term "individual" will be used throughout the document to denote the person who provides the responses to a data-collection instrument; the term is assumed to be synonymous with alternative terms such as "examinee" and "respondent," though it is acknowledged that each has domainspecific connotations. Along these lines, the data-collection instrument will be referred to as an "instrument," (representing alternative terms such as "test" and "questionnaire") and the term "trait" is used to denote the construct measured by the items (representing alternative terms such as "ability" and "proficiency"). Exceptions are made in rare instances when necessary to remain consistent with the literature. 2This initiative focuses on more accurate and efficient measurement of patient-reported symptoms and aspects of health-related quality of life. A primary goal of PROMIS is to develop instruments based on IRT methodologies in these domains that are publically available for the clinical research community.In IRT, estimates of respondents' traits (θ) are based not only on the responses they provide, but also the characteristics (i.e., parameters) of the items they are administered such as their difficulty -reflected by category boundary parameters (b) -their ability to differentiate among respondents -reflected by slope parameters (a) -and their susceptibility to guessing -reflected by lower asymptote parameters (c).One unidimensional IRT model frequently applied in health-outcomes settings is the graded response model (GRM;Samejima, 1969). The GRM is appropriate for item responses that fall in multiple ordered categories. It predicts the conditional probability of an individual responding in a particular category as a function of an individual's latent trait value and several item parameters.The GRM is considered a "difference model" (Thissen & Steinberg, 1986) or "indirect IRT model" (Embretson & Reise, 2000) because the probabilities are computed in two stages. Following Dodd, De Ayala, and Koch (1995), the probability that individual i will produce a response in category k or higher for item j is first computed as:where θ i is the individual's trait level, a j is the discrimination parameter for item j, and b jk is the category boundary for category k for item j. In the second step, the actual category 3 response probabilities for each response are computed by subtracting the cumulative probabilities from adjacent response categories conditional on θ using:There are a total of K -1 boundary parameters that need to be estimated for an item with K score categories. Figure 1 illustrates the probability of choosing each of the response options offered with an item from PROMIS designed to measure fatigue impact and experience according to the GRM. It shows that a person with a low level of fatigue would have a high probability of indicating that his fatigue made it "not at all hard to carry on a conversation." In ...

show abstract

Computer‐Based Testing and the Internet

Bartram¹

2017

The Blackwell Handbook of Personnel Selection

View full text Add to dashboard Cite

Operational Characteristics of Adaptive Testing Procedures Using the Graded Response Model

Cited by 57 publications

References 10 publications

Detection of Person Misfit in Computerized Adaptive Tests with Polytomous Items

Detection of Person Misfit in Computerized Adaptive Tests with Polytomous Items

Detecting Local Item Dependence in Polytomous Adaptive Data

Computer‐Based Testing and the Internet

Contact Info

Product

Resources

About