Investigating Linguistic Sources of Differential Item Functioning Using Expert Think-Aloud Protocols in Science Achievement Tests

Roth, Wolff‐Michael; Oliveri, María Elena; Sandilands, Debra; Lyons-Thomas, Juliette; Ercikan, Kadriye

doi:10.1080/09500693.2012.721572

Cited by 19 publications

(9 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This incomparability existed even between countries administering tests in the same language (Ercikan & McCreith, 2002;Ercikan et al, in press;Roth et al, 2013) and between language groups within countries (Ercikan et al, 2014;Kankaraš & Moores, 2013;Oliveri et al, 2012). It is important to identify whether item scores are comparable across groups since, if item scores are not comparable, the creation of a single scale score intended to represent all groups is not appropriate.…”

Section: Differential Item Functioning Analysesmentioning

confidence: 94%

Reading Proficiency and Comparability of Mathematics and Science Scores for Students From English and Non-English Backgrounds: An International Perspective

Ercikan

Chen

Lyons-Thomas

et al. 2014

International Journal of Testing

Self Cite

View full text Add to dashboard Cite

The purpose of this research is to examine the comparability of mathematics and science scores for students from English language backgrounds (ELB) and nonEnglish language backgrounds (NELB). We examine the relationship between English reading proficiency and performance on mathematics and science assessments in Australia, Canada, the United Kingdom, and the United States. The findings indicate a strong relationship with reading proficiency accounting for up to 43% of the variance in mathematics and up to 79% in science. In all comparisons, ELB students either outperformed NELB students or performed at the same level. However, when statistical adjustments were made for reading proficiency, in both mathematics and science, the score gap between the groups became statistically non-significant in three out of the four countries. These findings point to differences in score meaning in mathematics and science assessments and limitations in comparing performances of ELB and NELB.Correspondence should be sent to Kadriye Ercikan, 2125 Main Mall, ECPS,

show abstract

Section: Differential Item Functioning Analysesmentioning

confidence: 94%

Reading Proficiency and Comparability of Mathematics and Science Scores for Students From English and Non-English Backgrounds: An International Perspective

Ercikan

Chen

Lyons-Thomas

et al. 2014

International Journal of Testing

Self Cite

View full text Add to dashboard Cite

show abstract

“…We now propose ways in which experts can help inform the fairness review process in more detail. For instance, curricular, cultural, and linguistic experts can help identify possible sources of construct‐irrelevant variance for the major test‐taker populations prior to exporting an assessment (Roth, Oliveri, Sandilands, Lyons‐Thomas, & Ercikan, ). The group of reviewers may contain subject matter experts (e.g., appropriate faculty members, administrators, or both) to undertake a content review to determine whether the content and coverage of the tests are consistent with the content covered in the various curricula and aligned with the expectations of students majoring in the particular field(s) to which the test applies.…”

Section: Domain Analysis Evaluation and Generalization Componentsmentioning

confidence: 99%

The Validity of Inferences From Locally Developed Assessments Administered Globally

Oliveri

Lawless

2018

ETS Research Report Series

View full text Add to dashboard Cite

In this paper, we first examine the challenges of score comparability associated with the use of assessments that are exported. By exported assessments, we mean assessments that are developed for domestic use and are then administered in other countries in either the same or a different language. Second, we provide suggestions to better support their valid and fair use. We illustrate these issues in the context of higher education assessments that are designed to serve different purposes—inform admissions decisions and assess student learning outcomes within one country (e.g., the United States) that are later used in other countries. In higher education, the use of exported assessments is on the rise due to increases in globalization, student mobility, and cross‐national comparisons of student achievement. An increase in the use of exported assessments leads to more diverse test‐taker populations and requires special attention due to possible sources of construct‐irrelevant variance, which may threaten the score‐based inferences made for various populations. Irrelevant sources of variance may emerge due to differences in opportunity to learn, curricular exposure, and lack of familiarity with the cultural references used in the assessments that are exported.

show abstract

“…Ercikan et al (in press) investigated the impact of immediate and dominant culture environments for different language groups, and a number of studies have been completed by them (e.g., Ercikan et al, in press;Oliveri, Ercikan, & Zumbo, 2014;Roth, Oliveri, Sandilands, Lyons-Thomas, & Ercikan, 2013) that demonstrate how differential item functioning (DIF) results differ significantly by methods for ELs and differentially interact across groups of language minority students, kinds of environmental influences, and English language proficiency levels. Roth et al posited that at least some of the dissimilar DIF patterns may reflect different forms of reasoning.…”

Section: Grouped Relevant Accommodations For Els By Direct and Indirementioning

confidence: 99%

Advancing Accessibility and Accommodations in Content Assessments for Students With Disabilities and English Learners

Thurlow

Kopriva²

2015

Review of Research in Education

View full text Add to dashboard Cite

A ccessibility and accommodations in assessments now are seen as critical elements of an appropriately designed and implemented assessment of student achievement. The inclusion of all students in assessment systems designed to measure student knowledge and skills in various content areas has generated considerable and sometimes frenzied activity during the past two decades. This activity has taken place primarily at the state and national levels. With the funding of large consortia of states developing common assessments designed to measure college and career readiness, attention to accessibility and accommodations is being considered as well.The purpose of this chapter is to review what has happened, and what is likely to happen, as accessibility and accommodations research and practice for content assessments are advanced to ensure the appropriate inclusion and validity of assessment results for English learners (ELs) and students with disabilities. A description of accommodations and research specific to access for these two populations is beyond the scope and can be found in other chapters. Although we focus primarily on largescale content assessments used at the state, consortia, and national levels, much of what has happened in these assessments has direct implications for district content assessments and classroom-based assessments (e.g., online embedded tasks, publisher quizzes, teacher-developed quizzes, observational tools, performance tasks, etc.). Thus, we highlight some of those implications as well.Toward this end, we start by defining the many terms that have emerged around the topics of accessibility and accommodations. We address how these concepts have 556076R REXXX10.3102/0091732X14556076Review of Research in EducationThurlow, Kopriva: Advancing Accessibility and Accommodations research-article2014

show abstract

Investigating Linguistic Sources of Differential Item Functioning Using Expert Think-Aloud Protocols in Science Achievement Tests

Cited by 19 publications

References 19 publications

Reading Proficiency and Comparability of Mathematics and Science Scores for Students From English and Non-English Backgrounds: An International Perspective

Reading Proficiency and Comparability of Mathematics and Science Scores for Students From English and Non-English Backgrounds: An International Perspective

The Validity of Inferences From Locally Developed Assessments Administered Globally

Advancing Accessibility and Accommodations in Content Assessments for Students With Disabilities and English Learners

Contact Info

Product

Resources

About