Validating a Measure of Problem-Framing Ability to Support Evidence-Based Teaching Practice

Abstract

is a learning scientist and associate professor at the University of New Mexico in the Organization, Information & Learning Sciences program and in the Chemical & Biological Engineering Department. She served as Co-PI on an NSF RET Grant and a USDA NIFA grant, and is currently co-PI on three NSF-funded projects in engineering and computer science education, including a Revolutionizing Engineering Departments project. She was selected as a National Academy of Education / Spencer Postdoctoral Fellow and a 2018 NSF CAREER awardee in engineering education research. Dr. Svihla studies learning in authentic, real world conditions; this includes a two-strand research program focused on (1) authentic assessment, often aided by interactive technology, and (2) design learning, in which she studies engineers designing devices, scientists designing investigations, teachers designing learning experiences and students designing to learn.

A funds of knowledge approach to developing engineering students' design problem framing skills

“…Typically, students are provided with a realistic yet accessible scenario (Gulikers et al, 2008; Shah, 2005), and their work is evaluated using a rubric (Kan, 2007). We used a previously validated PBA, the Design Skills Test (DST), to measure students' design problem framing ability (L. White et al, 2019). In this approach, students are given 15 min in class to read and respond to a design brief detailing an accessible yet authentic problem.…”

“…We analyzed student work using the previously validated DST coding scheme (L. White et al, 2019), which focuses on design requirements (Table 4), design practices (Table 5), and design style (Table 6). We provide an example with an T A B L E 2 Design challenges developed with our first-year students' funds of knowledge in mind…”

“…We have not focused much discussion on design style as the two codes included in this dimension fall very short of serving as a comprehensive measure of design style, largely due to the complexity of what might be counted in this dimension, and is an area that deserves significant scholarly attention, especially in terms of measuring and tracking changes in how students develop and deploy their own judgments and preferences as they also develop as design engineers. Despite these limitations, we recognize and have argued elsewhere that the DST provides valid information for instructors about their design challenges but is just one tool in a larger toolkit (Svihla et al, 2020;L. White et al, 2019).…”

Insights and Outcomes from a Revolution in a Chemical Engineering Department

“…Had we begun this work a few years later, we would have incorporated other items [15,40]. We also developed a measure of design problem framing ability, a performance-based assessment that can be scored with a rubric to evaluate students' growth in problem framing skills [41].…”