Science assessments for all: Integrating science simulations into balanced state science assessment systems

Abstract

This article reports on the collaboration of six states to study how simulation‐based science assessments can become transformative components of multi‐level, balanced state science assessment systems. The project studied the psychometric quality, feasibility, and utility of simulation‐based science assessments designed to serve formative purposes during a unit and to provide summative evidence of end‐of‐unit proficiencies. The frameworks of evidence‐centered assessment design and model‐based learning shaped the specifications for the assessments. The simulations provided the three most common forms of accommodations in state testing programs: audio recording of text, screen magnification, and support for extended time. The SimScientists program at WestEd developed simulation‐based, curriculum‐embedded, and unit benchmark assessments for two middle school topics, Ecosystems and Force & Motion. These were field‐tested in three states. Data included student characteristics, responses to the assessments, cognitive labs, classroom observations, and teacher surveys and interviews. UCLA CRESST conducted an evaluation of the implementation. Feasibility and utility were examined in classroom observations, teacher surveys and interviews, and by the six‐state Design Panel. Technical quality data included AAAS reviews of the items' alignment with standards and quality of the science, cognitive labs, and assessment data. Student data were analyzed using multidimensional Item Response Theory (IRT) methods. IRT analyses demonstrated the high psychometric quality (reliability and validity) of the assessments and their discrimination between content knowledge and inquiry practices. Students performed better on the interactive, simulation‐based assessments than on the static, conventional items in the posttest. Importantly, gaps between performance of the general population and English language learners and students with disabilities were considerably smaller on the simulation‐based assessments than on the posttests. The Design Panel participated in development of two models for integrating science simulations into a balanced state science assessment system. © 2012 Wiley Periodicals, Inc. J Res Sci Teach 49: 363–393, 2012

Open Praxis vol. 9 issue 4

“…Indeed, this research is in line with those studies showing how the scientific education can be improved by the use of IT tools aimed to foster specific scientific cognitive skills, such as analysis and evaluation of phenomena in nature (Kirschner & Huisman, 1998). Research on the use of ITs in the learning and instruction of, for example, biology (e.g., Sewell, Stevens & Lewis, 1995), physics (Quellmalz, Timms, Silbergitt & Buckley, 2012), medicine (e.g., Bonnetain, Boucheix, Hamet & Freysz, 2010), can be found in the literature.…”

Creating an Indicator of K--12 Classroom Coverage of Science, Technology, Engineering, and Math (STEM) Content and Practices

“…They also developed ordered continua to characterize the complexity of scientific phenomena and the complexity of scientific practices, which they use to help in interpreting student interactions with the simulated environment. For example, the continuum for complexity runs from recognition at one end to developing arguments at the other (Quellmalz et al, 2012). The developers used evidence-centered design to incorporate levels of these continua into the design of the simulations.…”

Mācīšanās lietpratībai

“…Simulācijas var atkārtot, izmainīt mainīgos, rīkoties ar struktū-rām un modeļiem, kas citādi nebūtu pieejami. Tās mudina skolēnus izpētīt šo sistēmu veidojošo daļu mijiedarbību (Quellmalz, Timms, Buckley, Davenport, Loveland, & Silberglitt, 2012a;Quellmalz, Timms, Silberglitt, & Buckley, 2012b;Srisawasdi, & Kroothkeaw, 2014). Tiek uzsvērta simulāciju interaktivitātes nozī-mība, kā arī tas, ka simulācijas var izmantot kā neatkarīgu rīku, ar ko darbojas simulācijas veicējs, vai arī tajā var integrēt vērtēšanu (Neumann, 2010).…”

“…Līdzīga dabaszinātņu simulācija ir izmantota kā formatīvās vērtēšanas rīks, lai vērtētu dabaszinātnisko izpratību un apgūto zināšanu un prasmju atbilstību mācību priekšmeta standarta prasībām (Quellmalz et al, 2012a;Quellmalz et al, 2012b). Apstiprinoši rezultāti par formatīvās vērtēšanas izmantošanas lietderību ir iegūti arī citā pētījumā (Srisawasdi, & Kroothkeaw, 2014;Srisawasdi, & Sornkhatha, 2014).…”