IRTGEN: A SAS Macro Program to Generate Known Trait Scores and Item Responses for Commonly Used Item Response Theory Models

Whittaker, Tiffany A.; Fitzpatrick, Steven J.; Williams, Natasha; Dodd, Barbara G.

doi:10.1177/0146621603027004005

Cited by 24 publications

(21 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Missing values, due to program memory constraints, were assumed to be missing at random (MAR). In order to conduct the LLTM analysis, missing values were imputed using IRTGEN, a response generation program developed at the University of Texas (Whittaker, Fitzpatrick, Williams, & Dodd, 2003). The 1-PL estimated item and person parameters from the PARSCALE analysis were entered into the response generation program.…”

Section: Missing Datamentioning

confidence: 99%

Manipulating Processing Difficulty of Reading Comprehension Questions: The Feasibility of Verbal Item Generation

Gorin¹

2005

J Educational Measurement

View full text Add to dashboard Cite

Based on a previously validated cognitive processing model of reading comprehension, this study experimentally examines potential generative components of text‐based multiple‐choice reading comprehension test questions. Previous research (Embretson & Wetzel, 1987; Gorin & Embretson, 2005; Sheehan & Ginther, 2001) shows text encoding and decision processes account for significant proportions of variance in item difficulties. In the current study, Linear Logistic Latent Trait Model (LLTM; Fischer, 1973) parameter estimates of experimentally manipulated items are examined to further verify the impact of encoding and decision processes on item difficulty. Results show that manipulation of some passage features, such as increased use of negative wording, significantly increases item difficulty in some cases, whereas others, such as altering the order of information presentation in a passage, did not significantly affect item difficulty, but did affect reaction time. These results suggest that reliable changes in difficulty and response time through algorithmic manipulation of certain task features is feasible. However, non‐significant results for several manipulations highlight potential challenges to item generation in establishing direct links between theoretically relevant item features and individual item processing. Further examination of these relationships will be informative to item writers as well as test developers interested in the feasibility of item generation as an assessment tool.

show abstract

Section: Missing Datamentioning

confidence: 99%

Manipulating Processing Difficulty of Reading Comprehension Questions: The Feasibility of Verbal Item Generation

Gorin¹

2005

J Educational Measurement

View full text Add to dashboard Cite

show abstract

“…Item responses for a sample of 1,000 simulees were generated from the true generating item parameter values using the (Whittaker, Fitzpatrick, Williams, & Dodd, 2003). The true θ levels for each sample of simulees were drawn from a normal distribution with a mean of 0 and a standard deviation of 1.…”

Section: Item Pool Description and Data Generationmentioning

confidence: 99%

Comparing computer adaptive testing stopping rules under the generalized partial-credit model

et al. 2018

Self Cite

View full text Add to dashboard Cite

An important consideration of any computer adaptive testing (CAT) program is the criterion used for ending item administration-the stopping rule, which ensures that all examinees are assessed to the same standard. Although various stopping rules exist, none of them have been compared under the generalized partial-credit model (Muraki in Applied Psychological Measurement, 16, 159-176, 1992). In this simulation study we compared the performance of three variable-length stopping rules-standard error (SE), minimum information (MI), and change in theta (CT)-both in isolation and in combination with requirements of minimum and maximum numbers of items, as well as a fixed-length stopping rule. Each stopping rule was examined under two termination criteria-one a more lenient requirement (SE = 0.35, MI = 0.56, CT = 0.05), and one more stringent (SE = 0.30, MI = 0.42, CT = 0.02). The simulation design also included content-balancing and exposure controls, aspects of CAT that have been excluded in previous research comparing variable-length stopping rules. The minimum-information stopping rule produced biased theta estimates and varied greatly in measurement quality across the theta distribution. The absolute-change-in-theta stopping rule had strong performance when paired with a lower criterion and a minimum test length. The standard error stopping rule consistently provided the best balance of measurement precision and operational efficiency and was based on the fewest number of administered items necessary to obtain accurate and precise theta estimates, particularly when it was paired with a maximum-number-of-items stopping rule.

show abstract

“…Using the 157 item parameters based on the PCM, responses were generated on the basis of the IRTGEN SAS macro (Whittaker, Fitzpatrick, Williams, & Dodd, 2003). Fifty replications with 1,000 normally simulated examinees (0, 1) were generated and used for each condition for the present study.…”

Section: Mst Simulationmentioning

confidence: 99%

A comparison of panel designs with routing methods in the multistage test with the partial credit model

et al. 2013

Self Cite

View full text Add to dashboard Cite

In this study, we compared panel designs applied with various routing methods in the multistage test (MST) based on the partial credit model in the context of classification testing. Simulations were performed to compare three routing methods and four panel structures. Conditions of two test lengths and three passing rates were also included. The results showed that, regardless of the routing method used, the same panel structure performed similarly in terms of the precision of the classification decision with the same test length condition. The longer test length produced higher accuracy, whereas the 50 % passing rate yielded the lowest accuracy. Finally, all MST conditions performed well in terms of test security.

show abstract

IRTGEN: A SAS Macro Program to Generate Known Trait Scores and Item Responses for Commonly Used Item Response Theory Models

Cited by 24 publications

References 6 publications

Manipulating Processing Difficulty of Reading Comprehension Questions: The Feasibility of Verbal Item Generation

Manipulating Processing Difficulty of Reading Comprehension Questions: The Feasibility of Verbal Item Generation

Comparing computer adaptive testing stopping rules under the generalized partial-credit model

A comparison of panel designs with routing methods in the multistage test with the partial credit model

Contact Info

Product

Resources

About