Assessing Mechanistic Reasoning: Supporting Systems Tracing

Abstract

Reasoning about mechanism is central to disciplined inquiry in science and engineering and should thus be one of the foundations of a science, technology, engineering, and mathematics education. In addition, mechanistic reasoning is one of the core competencies listed in the Next Generation Science Standards (NGSS) Engineering Concepts and Practices (NGSS Lead States, 2013). Mechanistic explanations focus on the processes that underlie cause-effect relationships and consider how the activities of system components affect one another. While some assessment work has been accomplished in engineering education, to date mechanistic reasoning is an area where limited assessment development has been accomplished for pre-college populations. The data in this study come from the calibration of the Assessment of Mechanistic Reasoning Project (AMRP) (Weinberg, 2012), designed to diagnose individuals' mechanistic reasoning about systems of levers. This assessment presents a domain-specific characterization of mechanistic reasoning and illuminates what is easy and difficult about this form of reasoning. The study participants included elementary, middle, and high school students as well as college undergraduates and adults without higher education. Within this calibration study, item analyses, reliability, and validity measures were conducted using item response theory modeling; the AMRP assessment was found to have high reliability and validity. In addition, this study shows that machine characteristics such as number of levers, lever type, and arrangement of levers can affect the difficulty of mechanistic reasoning.

A Pathway Towards STEM Integration: Embodiment, Mathematization, and Mechanistic Reasoning

“…The literature suggests that mechanistic reasoning about even simple systems is not trivial for many children (or adults) (Lehrer & Schauble, 1998;Bolger et al, 2012;Weinberg, 2017a;2017b;2019). In Bolger et al's study, children predicted and explained the motion of pegboard linkages (Figure 1).…”

“…A challenge facing STEM education is integration, advancing student conceptual development within and across Science, Technology, Engineering, and Mathematics (STEM) domains (National Research Council [NRC], 2007). The National Academy of Engineering (NAE) (Honey, Pearson, & Schweingruber, 2014) has argued that integrated STEM education should bring together concepts from more than one discipline (e.g., mathematics and science; science, technology, and engineering); it may connect a concept from one domain to a practice of another, such as applying properties of geometric shapes (mathematics) to engineering design (Weinberg, 2017a;2017b;2019); or it may combine two practices, such as scientific inquiry (which includes modeling, argument, etc.) and engineering design (in which blueprints can be developed).…”

“…Reasoning about mechanism is foundational to disciplined inquiry in science and engineering; thus, it should be one of the foundations of a Science, Technology, Engineering, and Mathematics (STEM) education (Bolger, Kobiela, Weinberg, & Lehrer, 2012;National Research Council, 2011;Russ, Scherr, Hammer, & Mikeska, 2008;Weinberg, 2017a;2017b;2019). Individuals who engage in mechanistic explanations focus on the processes that undergird causal relationships and consider how the elements (and the relations between those elements) of system components affect one another.…”

“…In both of these studies, the majority of participants did not engage in mechanistic explanations. In Weinberg (2017a;2017b;2019), participants predicted and explained the motion of pegboard linkages represented on an assessment. Most children's mechanistic reasoning was fragmented, displaying few of the mechanistic elements necessary to describe lever motion.…”

“…Bolger et al (2012) noted that out of 72 total pegboard machines explained, there were only 6 instances in which a child recognized both constraint of the fixed pivot and rotation, indicating the difficulty of coordinating these two ideas. Weinberg (2017a;2017b;2019) developed the AMRP assessment with Item Response Theory (IRT) modeling. He showed that there were mean differences in difficulty between participants who did not have the propensity to diagnose the rotation of a system of levers (M = -0.67) and those who did (M = -0.26 logits, p < 0.1, one-tailed t-test).…”