Item Response Theory

Böck, Ronnie; Gibbons, Robert D.

doi:10.1002/9781119716723

Cited by 33 publications

(47 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In what follows, it is assumed that MLEs ν and their asymptotic covariance matrix cov(ν) are available (see e.g., Bock and Gibbons, 2021). The shorthand notation Y i = Y i (ν) is reserved for the sample-based quantities only.…”

Section: Differential Item Functioning Via Robust Scalingmentioning

confidence: 99%

“…Second, in the absence of DIF, the population model is deterministic (e.g., there is no residual term in Equation ( 2)). This implies that the only source of variation in the sample-based scaling problem is the (co-)variances of the item parameters estimates, which are available, for example, via known results on maximum likelihood estimation in IRT (e.g., Bock and Gibbons, 2021). Intuitively, these peculiar details of the identification problem in IRT provide an extraordinarily strong starting point from which to build up a methodology for DIF analysis.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Differential item functioning via robust scaling

Halpin¹

2022

Preprint

View full text Add to dashboard Cite

This paper proposes a new method for assessing differential item functioning (DIF) in item response theory (IRT) models. The method does not require pre-specification of anchor items, which is its main virtue. It is developed in two main steps, first by showing how DIF can be re-formulated as a problem of outlier detection in IRT-based scaling, then tackling the latter using established methods from robust statistics. The proposal is a redescending M-estimator of IRT scaling parameters that is tuned to flag items with DIF at the desired asymptotic Type I Error rate during estimation. Theoretical results guarantee that the method performs reasonably well when fewer than one-half of the items on a test exhibit DIF. Data simulations show that the proposed method compares favorably to currently available approaches, and a real data example illustrates its application in a research context where pre-specification of anchor items is infeasible. The focus of the paper is the twoparameter logistic model in two independent groups, with extensions to other settings considered in the conclusion.

show abstract

Section: Differential Item Functioning Via Robust Scalingmentioning

confidence: 99%

mentioning

confidence: 99%

Differential item functioning via robust scaling

Halpin¹

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Due to the same ecological approach mentioned above, missing data will always assume large proportions, since different student's need will dictate that each class will work on and assess different activities. IRT's algorithms were speci cally designed to work with categorical data and are robust to missing data, using all information available to estimates parameters that also have higher degrees of invariance from sample to sample (Bock & Gibbons, 2021;Reise & Revicki, 2015). In this way, students with just a few assessed activities will still be able to be scored.…”

Section: Strengths and Limitationsmentioning

confidence: 99%

Portuguese Physical Literacy Assessment Observation (PPLA-O) for adolescents (15-18 years) from grades 10-12: development and initial validation through Item Response Theory

Mota

Martins

Onofre

2022

Preprint

View full text Add to dashboard Cite

Background Aims of these studies were to develop the PPLA-Observation instrument (PPLA-O) to assess the physical and part of cognitive domain of Physical Literacy through data collected routinely by Physical Education (PE) teachers; and assess the construct validity (dimensionality, measurement invariance, and convergent and discriminant validity) and score reliability of one of its modules (Movement Competence, Rules and Tactics [MCRT]). Methods Content analysis of the Portuguese PE syllabus and literature review were used for domain identification of the PPLA-O. Multidimensional Item Response Theory (MIRT) models were used to assess construct validity and reliability, along with bivariate correlations in a sample of 515 Portuguese grade 10–12 students (Mage = 16, SD = 1). Results PPLA-O development resulted in an instrument with two modules: MCRT (22 physical activities) and Health-Related Fitness (5 protocols); both assessed with teacher-reported data entered in a spreadsheet. A two correlated dimensions Graded Response Model (Manipulative-based Activities [MA], and Stability-based Activities [SA]) showed best fit to the MCRT data, suggesting measurement invariance across sexes, and adequate to good score reliabilities (MA = .89, and SA = .73). There was a moderate to high correlation (r = .68) between dimensions, and boys had higher scores in both dimensions. Correlations among MCRT scores and HRF variables were similar in magnitude to previous reports in meta-analysis and systematic reviews. Conclusions The resulting PPLA-O is composed of two modules that integrate observational data collected by PE teachers into a common frame of criterion-referenced PL assessment. While the HRF module makes use of data collected through widely validated FITescola® assessment protocols, the MCRT makes use of teacher-reported data collected in a wide range of activities and movement pursuits to measure movement competence and inherent cognitive skills (Tactics and Rules). We also gathered initial evidence supporting construct validity and score reliability of the MCRT module. This highly feasible instrument can be used to provide Portuguese grade 10–12 (15–18 years) PE students with feedback on their PL journey, along with the other instrument of PPLA (PPLA-Q). Further studies should focus on assessment of inter and intra-rater reliability and criterion-related validity of its two modules.

show abstract

“…Alternatively, one could treat each individual response as an independent item that is either selected (1) or not selected (0) and use a dichotomous model for each (e.g., the twoparameter logistic, 2PL [13]); however, this assumption of independence may not be valid for MCMR responses, some of which may express opposite ideas (e.g., "Energy was added . .…”

Section: Introductionmentioning

confidence: 99%

Analyzing Multiple-Choice-Multiple-Response Items Using Item Response Theory

Smith¹,

Eaton²,

Brahmia³

et al. 2022

2022 Physics Education Research Conference Proceedings

View full text Add to dashboard Cite

Multiple-choice-multiple-response (MCMR) items allow students to select as many responses as they think are correct to a given question stem. Using MCMR items can provide researchers and instructors with a richer and more complete picture of what students do and do not understand about a particular topic. Interpreting students' MCMR responses is more nuanced than it is for single-response items. Unfortunately, many typical analyses of data from multiple-choice tests assume dichotomously-scored items, which eliminates the possibility of incorporating the rich information from students' response patterns to MCMR items. We present a novel methodology for using a combination of item response theory models to analyze data from MCMR items. These methods could be applied to inform scoring models that incorporate partial credit for various response patterns.2022 PERC Proceedings edited by Frank, Jones, and Ryan; Peer-reviewed, doi.org/10.1119/perc.2022.pr.Smith Published by the American Association of Physics Teachers under a Creative Commons Attribution 4.0 license.

show abstract

Item Response Theory

Cited by 33 publications

References 0 publications

Differential item functioning via robust scaling

Differential item functioning via robust scaling

Portuguese Physical Literacy Assessment Observation (PPLA-O) for adolescents (15-18 years) from grades 10-12: development and initial validation through Item Response Theory

Analyzing Multiple-Choice-Multiple-Response Items Using Item Response Theory

Contact Info

Product

Resources

About