Exploring the Feasibility of an Educational Computer Game as a Novel Means of Assessing Problem Solving Competencies

Abstract

IntroductionThis paper is grounded in the notion that educating problem solvers is one of the most critical functions of higher education. Indeed, governmental bodies and industry regularly demand such problem-solving competencies and they are becoming an increasingly intentional aspect of engineering education curricula.1,2 Despite the focus on problem-solving, a disconnect exists between the types of problems solved in educational settings and those encountered in professional settings and communities. 3 We believe this disconnect is in part because education systems are structured on assessment (e.g., grades, diplomas, accreditation) and thus do not prioritize teaching what cannot reliably be assessed. Operationalizing and rigorously assessing complex problem solving skills requires examining the reasoning process itself in addition to evaluating the accuracy of solutions generated. Most methods of researching problem-solving process involve self-report data which, while validated, still have limitations associated with the difference between individual perception and observable action. Building on the work of others to address such limitations, 4,5 we prioritize research protocols that move beyond pairing of achievement (right/wrong answers) and self-reported reasoning process (think-aloud protocols) to include observable data during problem-solving process that can corroborate (or refute) the self-report data.The central aim of this paper is to explore the feasibility of using an educational computer game as a novel means of assessing problem-solving competency. In this case, the novelty of the approach compared to traditional methods is twofold: (1) the use of a computer game enables us to directly observe problem-solving process through action in the game and (2) because of the naturally immersive game environment, we hope to see motivation and persistence in the face of complex problems that might otherwise be difficult to achieve in a laboratory setting. As a feasibility study, we will present methods and data that we suggest should guide further research but should not be used to make specific claims. Before describing this exploratory study, we briefly review relevant literature of problems and problem-solving. Background on Problem Solving Literature

Evaluating the Potential of fNIRS Neuroimaging to Study Engineering Problem Solving and Design

“…Because of its connection to STEM concepts, free versions are available for educational use along with community-sourced scenarios for specific purposes (ranging from secondary school common core standards to college freshman in physics). In pilot work not using fNIRS [23], we posed a series of in-game puzzles to students, observed their problem solving attempts, and conducted retrospective interviews. Artifacts from the activities were documented through video recordings of the computer screen which allowed us to study the diverse ways in which participants behave as they work towards solutions.…”

Towards Designing an Interactive System for Accelerated Learning and Assessment in Engineering Mechanics: A First Look at the Deforms Problem-solving System

“…We finally talk about creating unknowns and variable associations (6). Variable associations involve selecting multiple boxes in different equations to create systems of equations in more than one unknown.…”

“…The aim is to capture students' problemsolving processes, quickly assess correctness, and provide accurate feedback through targeted hints or explicit feedback. This approach accelerates and enhances learning by reducing algebraic tedium and emphasizing deliberate practice [4], which has been shown to enhance problem-solving accuracy and motivation to learn more [5], [6].…”

Decision-Making, Information Seeking, and Compromise: A Simulation Game Activity in Global Industrial Management

“…Simulation games-procedural representations of reality consisting of systems in which players engage in artificial conflict, defined by rules, that results in a quantifiable outcomes (Salen & Zimmerman, 2004)-represent one means of creating such activities. Scholars Gee and Hayes (2012) argue that games can be fundamentally understood as problem spaces, and the problems they contain can range from the well-structured spatial puzzles of Tetris to the ill-structured Rube Goldberg-type problems of Contraption Maker (Grohs et al, 2016). Moreover, these problems can be made sufficiently authentic if they are part of a simulation whose designers strive for high fidelity (Madhavan & Lindsay, 2014;Shuman, Besterfield-Sacre, & McGourty, 2005).…”