Validity Evidence for Learning Progression‐Based Assessment Items That Fuse Core Disciplinary Ideas and Science Practices

Abstract

This article evaluates a validity argument for the degree to which assessment tasks are able to provide evidence about knowledge that fuses information from a progression of core disciplinary ideas in ecology and a progression for the scientific practice of developing evidence-based explanations. The article describes the interpretive framework for the argument, including evidence for how well the assessment tasks are matched to the learning progressions and the methods for interpreting students' responses to the tasks. Findings from a dual-pronged validity study that includes a think-aloud analysis and an item difficulty analysis are presented as evidence. The findings suggest that the tasks provide opportunities for students at multiple ability levels to show evidence of both successes and struggles with the development of knowledge that fuses core disciplinary ideas with the scientific practice of developing evidence-based explanations. In addition, these tasks are generally able to distinguish between different ability-level students. However, some of the assumptions in the interpretive argument are not supported, such as the inability of the data to provide evidence that might neatly place students at a given level on our progressions. Implications for the assessment system, specifically, how responses are elicited from students, are discussed. In addition, we discuss the implications of our findings for defining and redesigning learning progressions. # 2013 Wiley Periodicals, Inc. J Res Sci Teach 50: 597-626, 2013

Design by taking perspectives: How engineers explore problems

“…The method allows for the examination of the design process in action (Akin, ) with minimal disruption of cognitive performance (Ericsson & Simon, ). As designers verbalize their thoughts as they occur, they may reveal what participants understand, what they pay attention to, and what guides them (Bernadowski, ; Bilda, Gero, & Purcell, ; Gotwals & Songer, ; Pergams, Jake‐Matthews, & Mohanty, ; Stieff, ; Wiltschnig, Christensen, & Ball, ). This think‐aloud method provides researchers with data not otherwise accessible about thoughts contributing to the creation of final explanations or solutions.…”

Key challenges and future directions for educational research on scientific argumentation

“…For best supporting students' learning of argumentation, there is a need to assess in a valid and reliable manner-both across the multiple dimensions of this science practice (e.g., product and process), and over time (Berland & McNeill, 2010;. There is also the question of how to integrate disciplinary core ideas with argumentation skills in assessment items (Gotwals & Songer, 2013). Furthermore, the presence of both conceptual and epistemic elements to student argumentation also complicates assessment efforts (Sandoval & Millwood, 2005) in terms of the role that science content plays when designing and interpreting assessments.…”

“…Other research focuses on supporting both pre-service (Erduran, Ardac, & Yakmaci-Guzel, 2006;Zembal-Saul, 2009) and in-service teachers (McNeill & Knight, 2013) in learning about and integrating argumentation into their instruction. Furthermore, there is work focused on assessment, such as assessing student engagement in argumentation (Gotwals & Songer, 2013;Osborne, Simon, Christodoulou, Howell-Richardson, & Richardson, 2013) and teachers' beliefs and pedagogical content knowledge for scientific argumentation Sampson & Blanchard, 2012).…”

Planning and carrying out investigations: an entry to learning and to teacher professional development around NGSS science and engineering practices

“…Songer has advanced the notion of 'fused' practices as a strategy for bundling together NGSS core ideas, crosscutting concepts, and science and engineering practices. In Songer et al (2009) and Gotwals and Songer (2013), the core idea biodiversity is blended with the crosscutting concept patterns and three fused practices: planning and carrying out investigations, analyzing and interpreting data, and constructing explanations. Rather than bundling practices, we advocate a practice unpacking stance.…”