Next-generation environments for assessing and promoting complex science learning.

Abstract

How can assessments measure complex science leaming? Although traditional, multiple-choice items can effectively measure declarative knowledge such as scientific facts or definitions, they are considered less well suited for providing evidence of science inquiry practices such as making observations or designing and conducting investigations. Thus, students who perform very proficiently in "science" as measured by static, conventional tests may have strong factual knowledge but little ability to apply this knowledge to conduct meaningful investigations. As technology has advanced, interactive, simulation-based assessments have the promise of capturing information about these more complex science practice skills. In the current study, we test whether interactive assessments may be more effective than traditional, static assessments at discriminating student proficiency across 3 types of science practices: (a) identifying principles (e.g., recognizing principles), (b) using principles (e.g., applying knowledge to make predictions and generate explanations), and (c) conducting inquiry (e.g., designing experiments). We explore 3 modalities of assessment: static, most similar to traditional items in which the system presents still images and does not respond to student actions, active, in which the system presents dynamic portrayals, such as animations, which students can observe and review, and interactive, in which the system depicts dynamic phenomena and responds to student actions. We use 3 analyses-a generalizability study, confirmatory factor analysis, and multidimensional item response theory-to evaluate how well each assessment modality can distinguish performance on these 3 types of science practices. The comparison of perfoimance on static, active, and interactive items found that interactive assessments might be more effective than static assessments at discriminating student proficiencies for conducting inquiry.

Digitale prøver – et litteraturstudie

“…beslutningstagning testes i en digital prøve bestående af cases og tilhørende spørgsmål om fx, hvilken undersøgelsesmetode man vil bruge i bestemte situationer. Quellmalz et al (2013) har arbejdet med interaktive, simulationsbaserede digitale prøver, og deres resultater tyder på, at MC--opgaver med bl.a. interaktive elementer kan anvendes til at undersøge eksaminandens evne til at identificere naturvidenskabelige principper.…”

A framework for designing and developing multimedia-based performance assessment in vocational education

“…And Levy (2013, p. 191) subtly remarks that: ''reflecting on the research conducted and lessons learned in the last 100 years with traditional assessment formats (…) begs the question whether a similar level wisdom is needed about how to best design simulations of assessment''. Finally, still other research (e.g., Quellmalz et al 2013;Halverson et al 2012;Wainess et al 2011;Vendlinski et al 2010;Shute et al 2009) presents parts of a methodology or principles for designing and developing MBPA and related types of assessment, but these were never integrated into a full design and development framework.…”

Multimedia-based performance assessment in Dutch vocational education

“…Our review of the literature yielded 10 studies that focused on product data, 6 studies that focused on process data, and 15 studies that focused on a combination of both (Table 3.3). An example of the first category is a study performed by Quellmalz et al (2013). Their SimScientists simulation was designed for measuring three science concepts in middle school science: identifying science principles, using science principles and conducting science inquiry.…”

“…Currently, much research is also being done on the use of simulations as assessments of science, physics, and biology concepts in middle school and high school. Among other simulations, SimScientists (Quellmalz, , 2013, Virtual Performance Assessment , ScienceAssistments (Gobert, 2012), Newton's Playground (Shute, 2013), and BioLogica focus on assessing science concepts.…”

“…For example, through the design and use of interactive tasks in an SBA, WestEd researchers were able to improve measurement of the conducting inquiry science practice in middle school (Quellmalz, Davenport, Timms, DeBoer, Jordan, Huang, & Buckley, 2013). Other researchers have even started to investigate the possibility to use SBA for very practical professions, for instance medical and security professions (Mislevy, Steinberg, Almond, Russell, Breyer, & Johnson, 2001;.…”