Investigating Sources of Differential Item Functioning in International Large-Scale Assessments Using a Confirmatory Approach

Sandilands, Debra; Oliveri, María Elena; Zumbo, Bruno D.; Ercikan, Kadriye

doi:10.1080/15305058.2012.690140

Cited by 20 publications

(10 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The effects of motivation upon test performances might differ across countries, as there are cultural differences in national pride and how it is enacted (Eklöf, 2010;Hopfenbeck and Kjaernsli, 2016). Additionally, the tests must be translated into different languages, which can impact upon the demands of the questions (Grisay, 2003;Wu and Ercikan, 2006;Grisay and Monseur, 2007;Wiliam, 2008;Hauger and Sireci, 2008;Ercikan and Koh, 2009;Grisay et al, 2009;Le, 2009;Solano-Flores, 2009;Arffman, 2010;Elvers, 2010;Babiar, 2011;Oliveri and Ercikan, 2011;Mesic, 2012;Sandilands et al, 2013;El Masri et al, 2016). Thus, many factors mean that using the data to compare countries to draw conclusions about how well students are learning in different education systems is not straightforward.…”

Section: Ilsa Barrier 4 -Comparisons Between Countries Are Problematicalmentioning

confidence: 99%

Assessment and learning: fields apart?

Baird

Andrich

Hopfenbeck

et al. 2017

Assessment in Education: Principles, Policy & Practice

187

108

View full text Add to dashboard Cite

Educational assessments define what aspects of learning will formally be given credit and therefore have a huge impact upon teaching and learning. Although the impact of high-stakes national and international assessments on teaching and learning is considered in the literature, remarkably, there is little research on the connection between theories of learning and educational assessments. Given the voluminous assessment that takes place annually in systematic ways in most many nations, it is surprising that more has not been gained from these assessments in the development of theories of learning and vice versa. In this article we consider both theories of learning and assessment and draw the main message of the article, that if assessments are to serve the goals of education, then theories of learning and assessment should be developing more closely with each other. We consider fundamental aspects of assessment theory, such as constructs, unidimensionality, invariance and quantifiability, and in doing so, we distinguish between educational and psychological assessment. Second, we show how less traditionally considered cases of a) international assessments and b) Assessment for Learning affect student learning. Through these cases we illustrate the otherwise somewhat theoretical discussion in the article. We argue that if assessment is to serve the learning goals of education, then this discussion on the relationship between assessment and learning should be developed further and be at the forefront of high-stakes, large-scale educational assessments.

show abstract

Section: Ilsa Barrier 4 -Comparisons Between Countries Are Problematicalmentioning

confidence: 99%

Assessment and learning: fields apart?

Baird

Andrich

Hopfenbeck

et al. 2017

Assessment in Education: Principles, Policy & Practice

187

108

View full text Add to dashboard Cite

show abstract

“…When type I error can be reasonably controlled under these conditions, applied researchers might prefer using the simple model (i.e., LR) for its ease of implementation and interpretation. A number of previous studies conducting DIF analysis on large-scale assessments ignored person clustering effect (e.g., Babiar 2011;Choi et al 2015;Hauger and Sireci 2008;Innabi and Dodeen 2006;Mahoney 2008;Mesic 2012;Ockey 2007;Oliveri et al 2014;Sandilands et al 2013). Therefore, evaluating the trade-off between complex versus simple modeling of DIF may provide supporting evidence for the findings of these studies.…”

Section: Person Clustering Effectmentioning

confidence: 52%

Comparing DIF methods for data with dual dependency

Jin¹,

Kang²

2016

Large-scale Assess Educ

View full text Add to dashboard Cite

“…In addition, even when items are pretested, because DIF could be a manifestation of an interaction between different factors distinguishing groups (Penfield & Camilli, 2007), DIF could still arise in the assessment. Indeed, data analysis of international large-scale assessments such as the Programme for International Student Assessment (PISA), which we illustrate in this study, provides empirical evidence of DIF across national versions of the same test (Asil & Brown, 2016;Ercikan & Koh, 2005;Grisay, de Jong, Gebhardt, Berezner, & Halleux-Monseur, 2007;Grisay, Gonzalez, & Monseur, 2009;Hauger & Sireci, 2008;Huang, Wilson, & Wang, 2016;Kreiner & Christensen, 2014;Le, 2009;Oliveri, Olson, Ercikan, & Zumbo, 2012;Sandilands, Oliveri, Zumbo, & Ercikan, 2013;Wu & Ercikan, 2006;Xie & Wilson, 2008). It is then common to resort to statistical methods for the post hoc accounting of DIF.…”

Section: Assessmentsmentioning

confidence: 82%

The Trade-Off between Model Fit, Invariance, and Validity: The Case of PISA Science Assessments

Masri

Andrich

2020

Applied Measurement in Education

View full text Add to dashboard Cite

In large-scale educational assessments, it is generally required that tests are composed of items that function invariantly across the groups to be compared. Despite efforts to ensure invariance in the item construction phase, for a range of reasons (including the security of items) it is often necessary to account for differential item functioning (DIF) of items post hoc. This typically requires a choice among retaining an item as it is despite its DIF, deleting the item, or resolving (splitting) an item by creating a distinct item for each group. These options involve a trade-off between model fit and the invariance of item parameters, and each option could be valid depending on whether or not the source of DIF is relevant or irrelevant to the variable being assessed. We argue that making a choice requires a careful analysis of statistical DIF and its substantive source. We illustrate our argument by analyzing PISA 2006 science data of three countries (UK, France and Jordan) using the Rasch model, which was the model used for the analyses of all PISA 2006 data. We identify items with real DIF across countries and examine the implications for model fit, invariance, and the validity of crosscountry comparisons when these items are either eliminated, resolved or retained.

show abstract

Investigating Sources of Differential Item Functioning in International Large-Scale Assessments Using a Confirmatory Approach

Cited by 20 publications

References 30 publications

Assessment and learning: fields apart?

Assessment and learning: fields apart?

Comparing DIF methods for data with dual dependency

The Trade-Off between Model Fit, Invariance, and Validity: The Case of PISA Science Assessments

Contact Info

Product

Resources

About