Assessing Mechanistic Reasoning

Abstract

Reasoning about mechanism is central to disciplined inquiry in science and engineering and should thus be one of the foundations of a science, technology, engineering, and mathematics education. In addition, mechanistic reasoning is one of the core competencies listed in the Next Generation Science Standards (NGSS) Engineering Concepts and Practices (NGSS Lead States, 2013). Mechanistic explanations focus on the processes that underlie cause-effect relationships and consider how the activities of system components affect one another.While some assessment work has been accomplished in engineering education, to date mechanistic reasoning is an area where limited assessment development has been accomplished for pre-college populations. The data in this study come from the calibration of the Assessment of Mechanistic Reasoning Project (AMRP) (Weinberg, 2012), designed to diagnose individuals' mechanistic reasoning about systems of levers. This assessment presents a domain-specific characterization of mechanistic reasoning and illuminates what is easy and difficult about this form of reasoning. The study participants included elementary, middle, and high school students as well as college undergraduates and adults without higher education. Within this calibration study, item analyses, reliability, and validity measures were conducted using item response theory modeling; the AMRP assessment was found to have high reliability and validity. In addition, this study shows that machine characteristics such as number of levers, lever type, and arrangement of levers can affect the difficulty of mechanistic reasoning.