Checking Equity: Why Differential Item Functioning Analysis Should Be a Routine Part of Developing Conceptual Assessments

Martinková, Patrícia; Drabinová, Adéla; Liaw, Yuan‐Ling; Sanders, Elizabeth A.; McFarland, Jenny; Price, Rebecca

doi:10.1187/cbe.16-10-0307

Cited by 93 publications

(89 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…males vs. females, or majorities vs. minorities. This so called differential item functioning [27] is a potential threat to item fairness and test validity and should therefore be tested routinely [28].…”

Section: A Psychometric Measures Used In Test and Item Evaluationmentioning

confidence: 99%

Semi-real-time analyses of item characteristics for medical school admission tests

Martinková

Štěpánek

Drabinová

et al. 2017

Proceedings of the 2017 Federated Conference on Computer Science and Information Systems

Self Cite

View full text Add to dashboard Cite

Abstract-University admission exams belong to so-called highstakes tests, i. e. tests with important consequences for the exam taker. Given the importance of the admission process for the applicant and the institution, routine evaluation of the admission tests and their items is desirable.In this work, we introduce a quick and efficient methodology and on-line tool for semi-real-time evaluation of admission exams and their items based on classical test theory (CTT) and item response theory (IRT) models. We generalize some of the traditional item analysis concepts to tailor them for specific purposes of the admission test.On example of medical school admission test we demonstrate how R-based web application may simplify admissions evaluation work-flow and may guarantee quick accessibility of the psychometric measures. We conclude that the presented tool is convenient for analysis of any admission or educational test in general.

show abstract

Section: A Psychometric Measures Used In Test and Item Evaluationmentioning

confidence: 99%

Semi-real-time analyses of item characteristics for medical school admission tests

Martinková

Štěpánek

Drabinová

et al. 2017

Proceedings of the 2017 Federated Conference on Computer Science and Information Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…Jejich výběr lze provést v záložce Data. Prvním z cvičných souborů je 20 položkový soubor "GMAT" (Martinková, Drabinová, Liaw, et al, 2017) obsahující data simulovaná na základě reálných parametrů z Graduate Admission Management Test (Kingston, Leary, & Wightman, 1985). První dvě položky v tomto souboru byly vygenerovány tak, aby fungovaly odlišně pro muže a ženy (odlišným 4 Viz https://shiny.rstudio.com/ 5 Viz https://shiny.cs.cas.cz/ShinyItemAnalysis/ 6 Viz https://cran.r-project.org/ 7 Jejich instalace se provede pomocí příkazu install.packages("Jméno chybějícího balíku") install.packages("ShinyItemAnalysis") library(ShinyItemAnalysis) startShinyItemAnalysis() TESTFÓRUM, 2017, č.…”

Section: Načtení Datunclassified

“…Naopak, potenciálně neférové položky mohou být přítomné i v případě, kdy se skupiny v celkovém počtu bodů neliší. V aplikaci se lze o této skutečnosti přesvědčit na datech "GMAT 2", která jsou simulována tak, že první dvě položky vykazují DIF, přestože rozdíl mezi skupinami v celkovém počtu bodů není signifikantní (pro případ, kdy jsou rozdělení celkových skóre dokonce zcela totožné, viz Martinková, Drabinová, Liaw, et al, 2017).…”

Section: Dif a Ddf Analýzaunclassified

See 1 more Smart Citation

ShinyItemAnalysis: Analýza přijímacích a jiných znalostních či psychologických testů

2017

Self Cite

View full text Add to dashboard Cite

“…In such a case, some aspect of the item, unrelated to the tested knowledge, could be unfairly causing the difference, thus DIF items are potentially unfair. Therefore, DIF analysis should be a routine part of a test and item validation process (Martinková, Drabinová, Liaw et al., ).…”

mentioning

confidence: 99%

Detection of Differential Item Functioning with Nonlinear Regression: A Non‐IRT Approach Accounting for Guessing

Drabinová

Martinková

2017

J Educational Measurement

Self Cite

View full text Add to dashboard Cite

In this article we present a general approach not relying on item response theory models (non‐IRT) to detect differential item functioning (DIF) in dichotomous items with presence of guessing. The proposed nonlinear regression (NLR) procedure for DIF detection is an extension of method based on logistic regression. As a non‐IRT approach, NLR can be seen as a proxy of detection based on the three‐parameter IRT model which is a standard tool in the study field. Hence, NLR fills a logical gap in DIF detection methodology and as such is important for educational purposes. Moreover, the advantages of the NLR procedure as well as comparison to other commonly used methods are demonstrated in a simulation study. A real data analysis is offered to demonstrate practical use of the method.

show abstract

Checking Equity: Why Differential Item Functioning Analysis Should Be a Routine Part of Developing Conceptual Assessments

Cited by 93 publications

References 57 publications

Semi-real-time analyses of item characteristics for medical school admission tests

Semi-real-time analyses of item characteristics for medical school admission tests

ShinyItemAnalysis: Analýza přijímacích a jiných znalostních či psychologických testů

Detection of Differential Item Functioning with Nonlinear Regression: A Non‐IRT Approach Accounting for Guessing

Contact Info

Product

Resources

About