Differential Item Functioning Detection With the Mantel-Haenszel Procedure: The Effects of Matching Types and Other Factors

Socha, Alan; DeMars, Christine E.; Zilberberg, Anna; Phan, Ha

doi:10.1080/15305058.2014.984066

Cited by 9 publications

(12 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We tried to examine whether the participants’ likelihood of choosing a determined value of an ORS item varied, or not, according to the group to which they belonged (i.e., reference group: men, or focal group: women). Given the fact that the items are ordinal, we observed the Mantel-Haenszel statistical test that is used as comparative criteria for the groups (Holland and Thayer, 1988, Socha et al, 2015), as well as the p value and its standard deviation (SDP). The significance of p -value was interpreted in order to examine the differences between groups in the observed item functioning, taking into account the significant values ( p < .05) as evidence that suggests the existence of DIF (González et al, 2011).…”

Section: Methodsmentioning

confidence: 99%

Factorial invariance, differential item functioning, and norms of the Orgasm Rating Scale

Arcos-Romero

Sierra

2019

International Journal of Clinical and Health Psychology

View full text Add to dashboard Cite

Background/Objective : Orgasm Rating Scale (ORS) assess the subjective orgasm experience in context of sexual relationship. It is composed of four dimensions attributed to the orgasm (Affective, Sensory, Intimacy, and Rewards). The purpose is to analyse the factorial invariance of the ORS across groups, to examine the metric equivalence across sex, and to present the standard scores. Method : A total of 1,472 Spanish adults (715 men and 757 women) were evaluated. They were distributed across age groups (18-34, 35-49 and 50 years old and older). Factorial invariance across different groups and the differential functioning of the items across sex were analyzed, internal consistency was examined, and the standard scores were developed. Results : The structure of the ORS showed strict measurement invariance across sex, relationship status, sexual orientation and education level. It also reached a scalar measurement invariance across age range and duration of the relationship. Some items showed a differential functioning between sexes. Conclusions : The Spanish version of the ORS is invariant across different groups at a factorial level, and it shows equivalence across sex in most of its items at a metric level. The standard scores allow a more accurate assessment of the subjective orgasm experience in context of sexual relationship.

show abstract

Section: Methodsmentioning

confidence: 99%

Factorial invariance, differential item functioning, and norms of the Orgasm Rating Scale

Arcos-Romero

Sierra

2019

International Journal of Clinical and Health Psychology

View full text Add to dashboard Cite

show abstract

“…Binning had been studied before in the literature on the use of MH D‐DIF procedures (Donoghue & Allen, 1993; Socha, DeMars, Zilberberg, & Phan, 2015; Zwick, Thayer, & Wingersky, 1995) for various reasons. The three binning methods we proposed are naive, linear, and equipercentile methods.…”

Section: Methodsmentioning

confidence: 99%

“…where ψ(θ) is the ability density distribution. This integration can be approximated via Gauss-Hermite quadratures (Socha et al, 2015;Steen, Byrne, & Gelbard, 1969).…”

Section: Appendix A: Computation Of Dif Measuresmentioning

confidence: 99%

Using Weighted Sum Scores to Close the Gap Between DIF Practice and Theory

Guo

Dorans

2019

J Educational Measurement

View full text Add to dashboard Cite

We make a distinction between the operational practice of using an observed score to assess differential item functioning (DIF) and the concept of departure from measurement invariance (DMI) that conditions on a latent variable. DMI and DIF indices of effect sizes, based on the Mantel-Haenszel test of common odds ratio, converge under restricted conditions if a simple sum score is used as the matching or conditioning variable in a DIF analysis. Based on theoretical results, we demonstrate analytically that matching on a weighted sum score can significantly reduce the difference between DIF and DMI measures over what can be achieved with a simple sum score. We also examine the utility of binning methods that could facilitate potential operational use of DIF with weighted sum scores. A real data application was included to show this feasibility.

show abstract

“…In a second step, a new sum score without the items that were identified to have DIF is computed for the final DIF analysis (Kok et al, 1985). When using iterative purification, this process is repeated until two runs yield the same DIF items (Kok et al, 1985;van der Flier et al, 1984) or until a maximum value of iterations is reached (Socha et al, 2015). Please note that while items that have previously been identified as DIF items are excluded from the matching criterion when ∆ MH is calculated, the item under investigation always contributes to the calculation of the matching criterion in this approach (Holland & Thayer, 1986;Raju et al, 1989;Zwick, 1990).…”

Section: Purification Of the Matching Criterionmentioning

confidence: 99%

“…There is a large number of studies comparing one or both purification strategies to a nonpurified approach (e.g., Clauser et al, 1993;Crane et al, 2006;DeMars, 2020;Fidalgo et al, 2000;French & Maller, 2007;Guilera et al, 2013;Hidalgo-Montesinos & Gómez-Benito, 2003;Khalid, 2011;Kwak et al, 1998;Navas-Ara & Gómez-Benito, 2002;Rogers & Swaminathan, 1993;Socha et al, 2015;Teresi, 2006;W.-C. Wang et al, 2009;W.-C. Wang & Su, 2004a, 2004b. Both, the 2-step as well as the iterative purification procedure have been shown to have higher DIF detection rates (Clauser et al, 1993;Fidalgo et al, 2000;Guilera et al, 2013), while iterative purification has the largest advantage over the other approaches when a large proportion of items have DIF (Fidalgo et al, 2000;French & Maller, 2007;Navas-Ara & Gómez-Benito, 2002).…”

Section: Purification Of the Matching Criterionmentioning

confidence: 99%

A new stopping criterion for Rasch trees based on the Mantel-Haenszel effect size measure for differential item functioning

Henninger¹,

Debelak²,

Strobl³

2021

Preprint

View full text Add to dashboard Cite

To detect differential item functioning (DIF), Rasch trees search for optimal splitpoints in covariates and identify subgroups of respondents in a data-driven way. To determine whether and in which covariate a split should be performed, Rasch trees use statistical significance tests. Consequently, Rasch trees are more likely to label small DIF effects as significant in larger samples. This leads to larger trees, which split the sample into more subgroups. What would be more desirable is an approach that is driven more by effect size rather than sample size. In order to achieve this, we suggest to implement an additional stopping criterion: the popular ETS classification scheme based on the Mantel-Haenszel odds ratio. This criterion helps us to evaluate whether a split in a Rasch tree is based on a substantial or an ignorable difference in item parameters, and it allows the Rasch tree to stop growing when DIF between the identified subgroups is small. Furthermore, it supports identifying DIF items and quantifying DIF effect sizes in each split. Based on simulation results, we conclude that the Mantel-Haenszel effect size further reduces unnecessary splits in Rasch trees under the null hypothesis, or when the sample size is large but DIF effects are negligible. To make the stopping criterion easy-to-use for applied researchers, we have implemented the procedure in the statistical software R. Finally, we discuss how DIF effects between different nodes in a Rasch tree can be interpreted and emphasize the importance of purification strategies for the Mantel-Haenszel procedure on tree stopping and DIF item classification.

show abstract

Differential Item Functioning Detection With the Mantel-Haenszel Procedure: The Effects of Matching Types and Other Factors

Cited by 9 publications

References 17 publications

Factorial invariance, differential item functioning, and norms of the Orgasm Rating Scale

Factorial invariance, differential item functioning, and norms of the Orgasm Rating Scale

Using Weighted Sum Scores to Close the Gap Between DIF Practice and Theory

A new stopping criterion for Rasch trees based on the Mantel-Haenszel effect size measure for differential item functioning

Contact Info

Product

Resources

About