Analysis of the most common concept inventories in physics: What are we assessing?

Abstract

Assessing student learning is a cornerstone of educational practice. Standardized assessments have played a significant role in the development of instruction, curricula, and educational spaces in college physics. However, the use of these assessments to evaluate student learning is only productive if they continue to align with our learning goals. Recently, there have been calls to elevate the process of science ("scientific practices") to the same level of importance and emphasis as the concepts of physics ("core ideas" and "crosscutting concepts"). We use the recently developed Three-Dimensional Learning Assessment Protocol to investigate how well the most commonly used standardized assessments in introductory physics (i.e., concept inventories) align with this modern understanding of physics education's learning goals. We find that many of the questions on concept inventories do elicit evidence of student understanding of core ideas, but do not have the potential to elicit evidence of scientific practices or crosscutting concepts. Furthermore, we find that the individual scientific practices and crosscutting concepts that are assessed using these tools are limited to a select few. We discuss the implications that these findings have on designing and testing curricula and instruction both in the past and for the future.

Evaluating the extent of a large-scale transformation in gateway science courses

“…for a substantial set of example 3DL assessment items, and to Underwood et al . ( 35 ) and Laverty and Caballero ( 36 ) for comparisons between traditional and 3DL assessment items]. In alignment with the intent of the Framework ( 9 ) that the three dimensions be integrated, we report the results in a binary way, that is, an assessment item either met the criteria for all three dimensions or it did not.…”

Forty Years’ Experience in Teaching Fluid Mechanics at Strasbourg University

“…Among the many tests developed for engineering and physics education in the United States [48][49][50][51][52][53][54][55], one is devoted to fluid mechanics [5,6]. Initiated by a cooperative effort between Jay Martin and John Mitchell at the University of Wisconsin-Madison and Ty Newell at the University of Illinois, Champaign-Urbana in 2001, it led to a final version in 2006 with the aim of promoting student understanding of fluid mechanics, as taught in mechanical engineering in the United States.…”

Characterizing lab instructors’ self-reported learning goals to inform development of an experimental modeling skills assessment

“…Such studies will help identify which instructional strategies are effective at improving students' modeling abilities in experimental physics contexts. 23. Do students enact revisions to the equipment, apparatus, or models?…”

“…In particular, there is a need to create instruments that measure students' competence with modeling. For example, Laverty and Caballero [23] showed that none of the four most widely used physics concept surveys target the concept of systems and system models or the practice of developing and using models. While there are a few ongoing efforts to assess some aspects of students' model-based reasoning in introductory physics lab and lecture courses [24][25][26], no modeling assessments exist for upper-division labs.…”