Construct Relevant or Irrelevant? The Role of Linguistic Complexity in the Assessment of English Language Learners’ Science Knowledge

Avenia-Tapper, Brianna; Llosa, Lorena

doi:10.1080/10627197.2015.1028622

Cited by 22 publications

(16 citation statements)

References 24 publications

(78 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unlike technical vocabulary-which are the specialized words specific to a discipline, academic vocabulary words are more generalized and span across many content areas (Stevens, Butler, & Castellon-Wellington, 2001). This distinction is important for many studies investigating the language effects of content assessments because while technical vocabulary is deemed to be construct-relevant, academic vocabulary is often seen as constructindependent and, subsequently, may interfere with the interpretations of student scores on assessments (Avenia-Tapper & Llosa, 2015;Haag, Roppelt, & Heppt, 2015;Wolf and Leon, 2009). Coxhead's (2000) academic word list (AWL) is used here to identify academic words on the assessment 3 .…”

Section: Item Features For the Learning Progressions In Middle Schoolmentioning

confidence: 99%

Exploring the item features of a science assessment with complex tasks

2018

View full text Add to dashboard Cite

Item explanatory models have the potential to provide insight into why certain items are easier or more difficult than others. Through the selection of pertinent item features, one can gather validity evidence for the assessment if construct-related item characteristics are chosen. This is especially important when designing assessment tasks that address new standards. Using data from the Learning Progressions in Middle School Science (LPS) project, this paper adopts an "item explanatory" approach and investigates whether certain item features can explain differences in item difficulties by applying an extension of the linear logistic test model. Specifically, this paper explores the effects of five features on item difficulty: type (argumentation, content, embedded content), scenario-based context, format (multiple-choice or open-ended), graphics, and academic vocabulary. Interactions between some of these features were also investigated. With the exception of context, all features had a statistically significant effect on difficulty.

show abstract

Section: Item Features For the Learning Progressions In Middle Schoolmentioning

confidence: 99%

Exploring the item features of a science assessment with complex tasks

2018

View full text Add to dashboard Cite

show abstract

“…Researchers have expressed concern that such tests may introduce additional English language demands that are not part of the constructs the tests are intended to measure, and that these demands may differentially affect EL test-takers (Abedi & Linquanti, 2012). Others have argued that the English language demands of science tests written in English are part of the construct that science tests are intended to measure, if the language of test items is consistent with the language of the discipline (Avenia-Tapper & Llosa, 2015). While we believe that some forms of language are appropriate targets of instruction, we do not believe that interpreting and producing complex academic language in English is the only way in which students can communicate about science, even though many testing regimes limit the ways students can demonstrate this ability.…”

Section: The Language Of Science Testsmentioning

confidence: 99%

Targeted Linguistic Simplification of Science Test Items for English Learners

Noble¹,

Sireci²,

Wells

et al. 2020

American Educational Research Journal

View full text Add to dashboard Cite

In this experimental study, 20 multiple-choice test items from the Massachusetts Grade 5 science test were linguistically simplified, and original and simplified test items were administered to 310 English learners (ELs) and 1,580 non-ELs in four Massachusetts school districts. This study tested the hypothesis that specific linguistic features of test items contributed to construct-irrelevant variance in science test scores of ELs. Simplifications targeted specific linguistic features, to identify those features with the largest impacts on ELs’ test performance. Of all the linguistic simplifications used in this study, adding visual representations to answer choices had the largest positive effect on ELs’ performance. These findings have significant implications for the design of multiple-choice test items that are fair and valid for ELs.

show abstract

“…A significant body of research reveals that high-stakes assessments present a challenge for ELs [6][7][8][9][10][11][12][13][14][15][16][17]. Research points to three significant issues that impact the assessment of ELs.…”

Section: Introductionmentioning

confidence: 99%

“…This assertion is supported by research that established a strong association between increasing linguistic complexity and decreasing test performance [7]. Second, a growing body of research demonstrates that ELs' test performance is reflective of their English language attainment versus their content knowledge [8][9][10]. As a case in point, in a study with 1700 ELs who were tested in English and Spanish on a standardized math achievement test, the results showed that the ELs answered more items correctly on a math test in their home language [11].…”

Section: Introductionmentioning

confidence: 99%

“…Researchers have established that there is a persistent disparity in test scores between ELs and non-ELs in academic content areas due to bias in testing [9,15]. However, there is a dearth of research on factors that impact EL test performance in general science and contentspecific strands-and subsequently, the implications for the validity of high-stakes science tests [16].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

High Quality, Equity, and Assessment: An Analysis of Variables Impacting English Learner Standardized Science Test Performance and Implications for Construct Validity

Salazar

Bruno

Schneider³

2022

Sustainability

View full text Add to dashboard Cite

In the United States, assessment is seen as a lever that can facilitate high-quality education. This study on English learners, students whose native language is not English, was based on data from eighth-grade English learners’ performance on science general and content-specific (physical, life, and earth science) standardized exams and an English language proficiency exam. The researchers utilized regression analysis to examine factors (i.e., socioeconomic status, home language, English language proficiency, and receptive and productive elements of language) that are predictive of English learner performance in general and content-specific science standardized assessments to identify implications for construct validity of high-stakes science assessments. The research question is as follows: What factors influence the performance of English learners on a standardized science assessment, including overall performance and content-specific domains? Three main findings emerged from this research study. First, this study confirms previous research indicating that socioeconomic status and English language proficiency are predictive of English learners’ achievement on content-based standardized tests. Second, this study adds to current research by providing evidence that productive language scores are the most significant predictors of English learner science achievement, in comparison to receptive language scores, overall English language proficiency scores, home language, and socioeconomic status. Third, this study adds to the body of evidence needed to challenge the validity of standardized science tests for English learner populations. The findings of this study challenge the construct validity of science content-based assessments for English learners, emphasizing the importance of productive language in academic performance.

show abstract

Construct Relevant or Irrelevant? The Role of Linguistic Complexity in the Assessment of English Language Learners’ Science Knowledge

Cited by 22 publications

References 24 publications

Exploring the item features of a science assessment with complex tasks

Exploring the item features of a science assessment with complex tasks

Targeted Linguistic Simplification of Science Test Items for English Learners

High Quality, Equity, and Assessment: An Analysis of Variables Impacting English Learner Standardized Science Test Performance and Implications for Construct Validity

Contact Info

Product

Resources

About