“I never thought of it as freezing”: How students answer questions on large‐scale science tests and what they know about science

Abstract

Education policy in the U.S. in the last two decades has emphasized large‐scale assessment of students, with growing consequences for schools, teachers, and students. Given the high stakes of such tests, it is important to understand the relationships between students' answers to test items and their knowledge and skills in the tested content area. Due to persistent test score gaps, students from historically non‐dominant communities, and their teachers and schools, are differentially affected by the consequences of large‐scale testing. As a result, it is particularly important to understand how students from historically non‐dominant communities interact with test items on large‐scale tests. We report on a study in which we interviewed 36 students about their responses to six multiple‐choice science test items from the Massachusetts state science assessment for fifth grade. The 36 students included 12 students from low‐income households, 12 English Language Learners, and 12 middle‐class native English speakers. We found that for five of the six selected test items, students' descriptions of the science content knowledge they used to answer the test items frequently did not match the content knowledge targeted by the items. In addition, students from low‐income households and English Language Learners were more likely than middle‐class native English speakers to answer incorrectly despite demonstrating knowledge of the targeted science content for the items. We argue that such evidence challenges the expectation that students' answers to individual test items reflect their knowledge of the targeted science content, and that evidence of this kind should be included in investigations of the validity of large‐scale tests. © 2012 Wiley Periodicals, Inc. J Res Sci Teach 49: 778–803, 2012

Science Assessments and English Language Learners: Validity Evidence Based on Response Processes

“…Analyzing student responses further, we found that nine of these 18 cases involved interpreting a question with the atypical perspective feature (either Take Away Heat or the other item in our set of six with this feature) instead of having the typical perspective. Although some non-ELL students also had difficulty with the atypical perspective feature of these two test items (see Noble et al, 2012), we found that ELL students' interpretations of the items with this feature were more likely to result in incorrect answers despite demonstrated target knowledge. Our results suggest that, while the atypical perspective feature may be intended to challenge students to view a scientific process from an alternative perspective, it can interfere with ELLs' interpretations of the language of the test item.…”

“…Additional questions were asked about vocabulary and illustrations specific to each item, and the interviewer asked follow-up questions when a student's answers were not clear or complete. A sample interview protocol is available in the online version of Noble et al (2012), along with a more detailed description of the methods used in this study. Interviews lasted approximately one hour and were videotaped and transcribed.…”

A closer look at linguistic complexity: Pinpointing individual linguistic features of science multiple-choice items associated with English language learner performance

“…Often, these assumptions privilege students who speak, read, and write English as a first language. In contrast, our prior research shows that the language of science test items may lead ELLs to construct alternative interpretations of the items and to choose incorrect answers, even in cases in which they know the science content being assessed (Noble, Rosebery, Suárez, Warren, & O'Connor, 2014;Noble et al, 2012). Therefore, it is important to examine how words and language structures function in testing contexts for students who are learning English as a second language.…”

Att kommunicera naturvetenskap i nationella prov : En studie med andraspråksperspektiv

“…Andra studier har visat att om inte bedömningen i form av prov, sker på samma språk som undervisningen bedrivs på, så uppstår en konflikt. Denna konflikt uppstår även om eleverna använder sitt förstaspråk frekvent utanför skolan (Noble et al, 2012;Okebukola, Owolabi, & Okebukola, 2013). Det har även visats att eftersom andraspråket ska användas som redskap i alla ämnen i senare skolår så bör det också vara undervisningsspråket för eleven.…”

“…Förstaspråket som ett språkligt redskap i en muntlig stöttning har i flera undersökningar visat sig gynna många elever i resonemang om uppgifter (Noble et al, 2012;Siegel, 2007). Att i ett samtal kunna byta språkdomän från en naturvetenskaplig till en mer vardaglig domän har visat sig vara en fruktsam strategi då uppgifter ska lösas (MackenHorarik, 1996).…”