Characterizing College Science Assessments: The Three-Dimensional Learning Assessment Protocol

Abstract

Many calls to improve science education in college and university settings have focused on improving instructor pedagogy. Meanwhile, science education at the K-12 level is undergoing significant changes as a result of the emphasis on scientific and engineering practices, crosscutting concepts, and disciplinary core ideas. This framework of “three-dimensional learning” is based on the literature about how people learn science and how we can help students put their knowledge to use. Recently, similar changes are underway in higher education by incorporating three-dimensional learning into college science courses. As these transformations move forward, it will become important to assess three-dimensional learning both to align assessments with the learning environment, and to assess the extent of the transformations. In this paper we introduce the Three-Dimensional Learning Assessment Protocol (3D-LAP), which is designed to characterize and support the development of assessment tasks in biology, chemistry, and physics that align with transformation efforts. We describe the development process used by our interdisciplinary team, discuss the validity and reliability of the protocol, and provide evidence that the protocol can distinguish between assessments that have the potential to elicit evidence of three-dimensional learning and those that do not.

Analysis of the most common concept inventories in physics: What are we assessing?

“…To help identify and develop assessments that are capable of eliciting evidence of students engaging with each of the three dimensions, we developed the ThreeDimensional Learning Assessment Protocol [27].…”

“…The 3D-LAP uses individual questions or clusters of related questions (referred to herein as a "task") as the unit of analysis. By analyzing only the task itself, the 3D-LAP can be used to determine if the task has the potential to elicit evidence that a student will engage in a scientific practice, crosscutting concept, or core idea [27].…”

“…While this lens backgrounds a number of important issues (e.g., epistemological development and shifts in identity), it foregrounds engaging students in the process of science (scientific practices) and helping students develop how they organize their knowledge (core ideas and crosscutting concepts). Our analysis makes use of the recently developed Three-Dimensional Learning Assessment Protocol (3D-LAP) [27], a tool that evidences how well assessments provide opportunities to engage students in 3DL. Using the 3D-LAP, we coded the questions appearing on the four most common concept inventories [Force Concept Inventory (FCI), Force and Motion Conceptual Evaluation (FMCE), Brief Electricity and Magnetism Assessment (BEMA), Conceptual Survey of Electricity and Magnetism (CSEM)] to determine the degree to which they can provide evidence of 3DL.…”

“…V), but defers to Ref. [27] for details, and analyzes the four most common standardized assessments in physics using the 3D-LAP (Sec. VI).…”

The Design and Transformation of Biofundamentals: A Nonsurvey Introductory Evolutionary and Molecular Biology Course

“…In addition, questions raised during in-class discussions were noted and used to inform subsequent revisions of first the website and then the text and associated beSocratic activities. Similarly, exam questions were analyzed using a rubric (three-dimensional learning assessment protocol [3D]-LAP) generated in the course of American Association of Universities (AAU)-funded studies at Michigan State University to characterize the nature of exam questions on the basis of a number of criteria, including whether the answers involved disciplinary core ideas and cross-cutting concepts and practices (Laverty et al ., 2016); these rubrics are illustrated for the final exam given in 2015 (see Supplemental Material 3). The course was routinely taught with the aid of learning assistants (Otero, 2006), undergraduate students who had previously taken the course, who were taking a concurrent course in science pedagogy, and who met weekly with the course instructor to discuss students’ issues.…”