Measuring learning in serious games: a case study with structural assessment

Abstract

The effectiveness of serious games is often measured with verbal assessment. As an alternative we propose Pathfinder structural assessment (defined as measuring the learners' knowledge organization and compare this with a referent structure) which comprises three steps: knowledge elicitation, knowledge representation and knowledge evaluation. We discuss practical and theoretical considerations for the use of structural assessment and showcase its application with the game Code Red: Triage. Results suggest that structural assessment measures an individual's understanding of a domain at least differently from verbal assessment. While verbal assessment may provide a more nuanced picture regarding declarative and procedural knowledge, structural assessment may add an in-depth understanding of the concepts that are regarded important in a domain. In the Discussion we propose four guidelines to effectively use structural assessment in serious games: (1) Determine the appropriateness of the domain for structural assessment, (2) select an appropriate referent for the target group(s), (3) select the number of concepts needed for structural assessment, and (4) consider the analysis of the graphical knowledge representations to obtain in-depth information about the quality of the knowledge structures.

Improving the Credibility of Child Sexual Assault Victims in Court: The Impact of a Sexual Assault Nurse Examiner

“…To this end, we quantified similarity between PFNETs from the three experimental conditions using a set theoretical measure closeness ( C ). C reflects the degree to which a concept has the same neighbors in two different PFNETs (Goldsmith & Davenport, ) and is a valid measurement of similarity between cognitive representations of knowledge (e.g., Wouters, Van der Spek, & Van Oostendorp, ). Values of C can theoretically vary from 0 to 1, with higher values of C indicating higher similarity between structures.…”

Games on Mobiles via Web or Virtual Reality Technologies: How to Support Learning for Biomedical Laboratory Science Education

“…SSGs, similar to other technologies, must be carefully chosen and adjusted to the specific learning situation [11,44], and from the very start, responsible teachers and instructors should know how to evaluate their effectiveness [9]. It is important to keep in mind that SSGs are not only tools that need to be adjusted and used, but tools and applications that need to be continually developed.…”

“…To fully utilize digital or serious games, important limitations could include the following: lack of knowledge about how to align the aims of the games to the learning goals, unintended side effects from the digital transition [7], or missing game elements that allow for in-depth learning [8]. This is mainly due to difficulties concerning how to systematically introduce and assess effects [9] or illustrate empirical benefits [10]. Tsekleves and his colleagues stated the following: "it is yet unclear how serious games can be best incorporated within formal education systems" [11].…”

Lean Product Development: Serious Game and Evaluation of the Learning Outcomes

“…In general, the application of games with the aim of education and learning is defined as "Serious games" [5]. In SG, players assume different roles and are involved in simple and complicated decision making processes, which makes it attractive for SBCE process where alternative design exploration and convergence involve multiple-views.…”