Predicting students' knowledge after playing a serious game based on learning analytics data: A case study

Abstract

Serious games have proven to be a powerful tool in education to engage, motivate, and help students learn. However, the change in student knowledge after playing games is usually measured with traditional (paper) prequestionnaires–postquestionnaires. We propose a combination of game learning analytics and data mining techniques to predict knowledge change based on in‐game student interactions. We have tested this approach in a case study for which we have conducted preexperiments–postexperiments with 227 students playing a previously validated serious game on first aid techniques. We collected student interaction data while students played, using a game learning analytics infrastructure and the standard data format Experience API for Serious Games. After data collection, we developed and tested prediction models to determine whether knowledge, given as posttest results, can be accurately predicted. Additionally, we compared models both with and without pretest information to determine the importance of previous knowledge when predicting postgame knowledge. The high accuracy of the obtained prediction models suggests that serious games can be used not only to teach but also to measure knowledge acquisition after playing. This will simplify serious games application for educational settings and especially in the classroom easing teachers' evaluation tasks.

Learning immunology in a game: Learning outcomes, the use of player characters, immersion experiences and visual attention distributions

“…The effect of the interaction between the player and the use of the PCs was reflected further in science learning outcomes, such that the less characters of the first defence line were used, the more characters of the third defence line were used and the better were the science learning outcomes. The results aligned with previous studies by Alonso‐Fernández et al (2020), Cheng, Lin, and She (2015) and Cheng, She, and Annetta (2015), supporting the value of the interactions with game characters in predicting game‐based learning outcomes. These together implied that Humunology 's game design is consistent with the actual immune mechanisms, so that the gameplay behaviours represent scientific facts well and are effective for learning overall.…”

“…Their another study extracted gameplay data, such as whether the game has been completed, game scores, the interactions with game elements and so forth to build models for predicting knowledge acquisition. The results demonstrated that gameplay data can be effective for accurately predicting student learning, and the number of interactions with game character was one best predictor (Alonso‐Fernández et al, 2020). Cheng, Lin, and She (2015) created an educational game for the students to learn the mechanism of biological evolution by manipulating a team of the PCs to compete with NPCs controlled by the computer.…”

“…Recently, there has been an emerging interest in analysing gameplay data by using various techniques (Loh, Sheng, & Ifenthaler, 2015a). The analyses of in‐game interactions by gameplay data can further help understand the way the players learn using games and determine whether the game design is consistent with learning content (Alonso‐Fernández, Martínez‐Ortiz, Caballero, Freire, & Fernández‐Manjón, 2020), which is important information to support that true learning really occurs in the gaming context. Therefore, one purpose of this study was to investigate the way the players interact with the game design of Humunology to reveal the stories behind the scenes by using gameplay data.…”

Multimodal learning analytics for game‐based learning

“…A number of studies have examined knowledge acquisition with game‐based learning environments using a range of data channels such as gameplay behavior traces (eg, Alonso‐Fernández et al ., 2019; Alonso‐Fernández, Martínez‐Ortiz, Caballero, Freire, & Fernández‐Manjón, 2020; Taub et al ., 2017), facial expressions of emotions (eg, Lane & D’Mello, 2019; Taub, Sawyer, Smith, et al ., 2020), performance measures (eg, pre/posttest scores; Dever & Azevedo, 2019), self‐report measures (eg, Cloude, Taub, Lester, & Azevedo, 2019) and eye gaze (eg, Dever, Wiedbusch, & Azevedo, 2019; Gomes, Yassine, Worsley, & Blikstein, 2013; Lee, Donkers, Jarodzka, & van Merriënboer, 2019; Tsai, Huang, Hou, Hsu, & Chiou, 2016). A noteworthy study used gameplay behavior traces to model students’ self‐reported interest during game‐based learning (Sawyer, Rowe, Azevedo, & Lester, 2018).…”

Simva: Simplifying the Scientific Validation of Serious Games

“…Simva has helped us in different validation experiments to simplify data collection and processing, saving us time and automating some of the processes that were most prone to errors. We have used Simva with more than 1000 students and 3 different serious games: in Conectado [2] we linked questionnaires with the collected LA data; in First Aid Game [8] we linked together data from two experiments over time; and in the 15 Objects test [9] we used the metadata feature to include the information of which of the two versions of the game each student was using.…”