Differential item functioning (DIF) of test items should be evaluated using practical methods that can produce accurate and useful results. Among a plethora of DIF detection techniques, we introduce the new Residual DIF (RDIF) framework, which stands out for its accessibility without sacrificing efficacy. This framework consists of three item response theory (IRT) residual statistics: RDIFR$RDI{F_R}$, RDIFS$RDI{F_S}$, and RDIFRS$RDI{F_{RS}}$. We conducted a simulation study with a 40‐item test to assess the performance of RDIF in comparison with the Mantel‐Haenszel, logistic regression, and IRT‐based likelihood ratio test methods. Even when analyzing small sample sizes, the results revealed RDIFRS$RDI{F_{RS}}$ to be the most robust DIF detection statistic with strict control of Type I error across all simulated conditions when paired with the purification procedure. Also, RDIFR$RDI{F_R}$ and RDIFS$RDI{F_S}$ proved to be powerful indicators of uniform and nonuniform DIF, respectively. Therefore, RDIFRS$RDI{F_{RS}}$ should serve as the primary flagging criterion, whereas RDIFR$RDI{F_R}$ and RDIFS$RDI{F_S}$ best serve as indicators of DIF type. An empirical DIF study also showed that the RDIF framework could perform satisfactorily with real data from a large‐scale assessment. Overall, the RDIF framework demonstrated its potential as a new standard for IRT‐based DIF detection methodology.
The Trends in International Mathematics and Science Study (TIMSS) makes it possible to compare the performance of students in the US in Mathematics and Science to the performance of students in other countries. TIMSS uses four international benchmarks for describing student achievement: Low, Intermediate, High, and Advanced. In this study, we linked the eighth-grade Math TIMSS and NAEP scales using equipercentile equating to (a) help better interpret U.S. eighth-grade students’ performance on TIMSS, and (b) investigate the progress of eighth-grade U.S. students over time relative to the progress of students in other countries. Results indicated that relative to other countries, U.S. eighth-grade students increased with respect to the “At or Above Basic” NAEP Achievement level, but that other countries saw larger improvements in the higher achievement level categories, relative to the US. This finding may reflect the emphasis of No Child Left Behind on raising lower achievement to “proficient.” However, with respect to “Advanced” mathematics achievement, eighth-grade U.S. students showed less improvement than students in other countries.
The R package irtplay provides practical tools for unidimensional item response theory (IRT) models that conveniently enable users to conduct many analyses related to IRT. For example, the irtplay includes functions for calibrating online items, scoring test-takers’ proficiencies, evaluating IRT model-data fit, and importing item and/or proficiency parameter estimates from the output of popular IRT software. In addition, the irtplay package supports mixed-item formats consisting of dichotomous and polytomous items.
Item response theory (IRT) is a general framework in which mathematical models are formulated to explain the relationship between an examinee's observable response on an item and the latent ability measured by a test. The application of IRT models and related statistical methods are commonly found in educational and psychological research. An important step in applying IRT models to test data is estimating the IRT model parameters. Accordingly, the successful application of IRT rests on the satisfactory statistical techniques and software for accurately estimating the model parameters.The irtplay R package was developed to provide users with a user-friendly experience and convenience when analyzing test data using unidimensional IRT models. The package can be used to fit the IRT models to a mixture of dichotomous and polytomous item data using marginal maximum likelihood estimation via the expectation-maximization, calibrate pretest items, and estimate examinees' latent ability parameters. In addition, the package provides practical tools that conveniently enable users to conduct many analyses related to IRT such as evaluating IRT model-data fit, analyzing differential item functioning, computing asymptotic variance-covariance matrices of item parameter estimates, calculating the conditional probability distribution of observed scores using the Lord and Wingersky (1984) formula, and importing item and ability parameter estimates from the output of popular IRT software. The main features of the irtplay package are illustrated using three data examples.
This case study applied the weak theory of Automatic Item Generation (AIG) to generate isomorphic item instances (i.e., unique but psychometrically equivalent items) for a large-scale assessment. Three representative instances were selected from each item template (i.e., model) and pilot-tested. In addition, a new analytical framework, differential child item functioning (DCIF) analysis, based on the existing differential item functioning statistics, was applied to evaluate the psychometric equivalency of item instances within each template. The results showed that, out of 23 templates, nine successfully generated isomorphic instances, five required minor revisions to make them isomorphic, and the remaining templates required major modifications. The results and insights obtained from the AIG template development procedure may help item writers and psychometricians effectively develop and manage the templates that generate isomorphic instances.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.