Inconsistencies in DIF Detection for Sub-Groups in Heterogeneous Language Groups

Ercikan, Kadriye; Roth, Wolff‐Michael; Simon, Marielle; Sandilands, Debra; Lyons-Thomas, Juliette

doi:10.1080/08957347.2014.944306

Cited by 26 publications

(9 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Research on large-scale assessment for English language learners (ELLs), which has implications for Native students, has increasingly pointed to the need to tailor assessments to subpopulations of ELLs to obtain valid scores on academic tests (Ercikan et al, 2014;Kopriva et al, 2016). One way of addressing this need is through computerized systems (Kopriva, 2014), but such systems are costly to develop and likely a long way off.…”

Section: Large-scale Testing Woesmentioning

confidence: 99%

The Ongoing Quest for Culturally-Responsive Assessment for Indigenous Students in the U.S.

Trumbull

Nelson‐Barber

2019

Front. Educ.

View full text Add to dashboard Cite

Section: Large-scale Testing Woesmentioning

confidence: 99%

The Ongoing Quest for Culturally-Responsive Assessment for Indigenous Students in the U.S.

Trumbull

Nelson‐Barber

2019

Front. Educ.

View full text Add to dashboard Cite

“…Accuracy is the level of accuracy and accuracy in measurement. According to Ercikan, Roth, Simon, Sandilands, and Lyons-Thomas (2014) accuracy is the ability to correctly identify the correct DIF items, while not identifying items that are not DIF.…”

Section: Accuracymentioning

confidence: 99%

Sensitivity and Accuracy of the Mantel-Haenszel Method and Standardization Method: Detection of Item Functioning Differential

Rustam

Naga

Supriyati

2019

IJELS

View full text Add to dashboard Cite

Detection of differential item functioning (DIF) is needed in the development of tests to obtain useful items. The Mantel-Haenszel method and standardization are tools for DIF detection based on classical theory assumptions. The study was conducted to highlight the sensitivity and accuracy between the Mantel-Haenszel method and the standardization method in DIF detection. Simulation design (a) test participants consisted of 1000 responses in both the reference and focus groups, (b) the size of the proportion of DIF (0.1; 0.25; 0.50; and 0.75), and (c) the length of the multiple choice test with 40 choices the answer. Research shows that the Mantel-Haenszel method has the same sensitivity as the standardization method in DIF proportions of 10% and 25%, however, when the ratio of DIF proportions above 25% the standardization method is less sensitive, and conversely the sensitivity of the Mantel-Haenszel method increases. The standardization method has higher accuracy than the Mantel-Haenszel method in the DIF proportion of 10%, however, when the size of the DIF proportion above 10% the accuracy of the standardization method decreases, the accuracy of the Mantel-Haenszel method is higher than the standardization method. Thus, if the ratio of DIF is detected by the standardization method of (≤0.10), then the results of the standardization method are preferred as a reference. Conversely, if the proportion of DIF detected by the standardization method is (≥0.10), then the result of the Mantel-Haenszel method is chosen as a reference.

show abstract

“…Existe evidencia de que cuanto más heterogéneo lingüísticamente y culturalmente es el grupo focal, es menos probable identificar ítems que funcionan diferencialmente (Ercikan, Roth, Simon, Sandilands & Lyons-Thomas, 2014). Una limitada capacidad evaluativa puede obstaculizar la capacidad de algunos países para examinar el funcionamiento diferencial de ítems para grupos focales importantes y para modelar adecuadamente la heterogeneidad de sus poblaciones.…”

Section: │12unclassified

Capacidad evaluativa, validez cultural, y validez consecuencial en PISA

Solano‐Flores

Milbourn

2016

RELIEVE

View full text Add to dashboard Cite

ResumenLas evaluaciones internacionales de estudiantes han desempeñado un papel cada vez más importante en la política educativa. Estas comparaciones internacionales basadas en pruebas generan información valiosa sobre el rendimiento del estudiantado de cada país participante y los factores sociales y contextuales asociados. Una imagen compleja de los factores culturales, económicos y sociales que dan forma a la participación de PISA empieza a emerger. Nuestro objetivo es entender la relación entre la capacidad evaluativa nacional y la forma en que los países participan en estas comparaciones internacionales. Proponemos un marco conceptual para examinar la capacidad evaluativa como clave para abordar dos aspectos de la validez: cultural y consecuencial. Asimismo, se discuten las múltiples facetas de la capacidad evaluativa como condiciones para abordar la validez cultural y validez consecuencial en comparaciones internacionales. Palabras clave:PISA; capacidad evaluativa; validez cultural; validez consecuencial AbstractInternational student assessments have played an increasing important role in educational policy. These international test comparisons generate valuable information about each participating country's student performance and the social and contextual factors. A complex picture of the cultural, economic, and social factors that shape PISA participation is emerging. We aim to understand the relationship between national assessment capacity and how countries participate in international test comparisons. We propose a framework for examining assessment capacity as key to addressing two aspects of validity -cultural and consequential. Also, we discuss the multiple facets of assessment capacity as conditions for addressing cultural validity and consequential validity in international test comparisons

show abstract

Inconsistencies in DIF Detection for Sub-Groups in Heterogeneous Language Groups

Cited by 26 publications

References 21 publications

The Ongoing Quest for Culturally-Responsive Assessment for Indigenous Students in the U.S.

The Ongoing Quest for Culturally-Responsive Assessment for Indigenous Students in the U.S.

Sensitivity and Accuracy of the Mantel-Haenszel Method and Standardization Method: Detection of Item Functioning Differential

Capacidad evaluativa, validez cultural, y validez consecuencial en PISA

Contact Info

Product

Resources

About