Exploring educational immersive videogames: an empirical study with a 3D multimodal interaction prototype

Abstract

Gestural interaction devices emerged and originated various studies on multimodal human-computer interaction to improve user experience. However, there is a knowledge gap regarding the use of these devices to enhance learning.We present an exploratory study which analysed the user experience with a multimodal immersive videogame prototype, based on a Portuguese historical/cultural episode. Evaluation tests took place in high school environments and public videogaming events. Two users would be present simultaneously in the same virtual reality environment: one as the helmsman aboard Vasco da Gama's XV-century Portuguese ship, another as the mythical Adamastor stone giant at the Cape of Good Hope. The helmsman player wore a virtual reality headset to explore the environment, whereas the giant player used body motion to control the giant, and observed results on a screen, with no headset. This allowed a preliminary characterization of user experience, identifying challenges and potential use of these devices in multi-user virtual learning contexts. We also discuss the combined use of such devices, towards future development of similar systems, and its implications on learning improvement through multimodal human-computer interaction.

A review of the use of virtual reality head-mounted displays in education and training

“…Findings like these all point towards the affordances of immersive technologies such as HMDs. This however, is somewhat contrasted by Fernandes et al (2016), who found that increasing immersion by adding 3D sounds and a graphical rendering of user's own hands to the virtual environment confused some study participants and distracted from the learning task.…”

“…Of the 21 studies 14 examined the learner experience (Andreoli et al, 2016;Bharathi & Tucker, 2015;Fernandes et al, 2016;Gutiérrez-Maldonado, Ferrer-García, Plasanjuanelo, Andrés-Pueyo, & Talarn-Caparrós, 2015;Janssen, Tummel, Richert, & Isenhardt, 2016;Kleven et al, 2014;Loup, Serna, Iksal, & George, 2016;Madrigal, Prajapati, & Hernandez-Prera, 2016;Moesgaard et al, 2015;Pan et al, 2016;Polcar & Horejsi, 2015;Ray & Deb, 2016;Reiners, Wood, & Gregory, 2014;Stavroulia et al, 2016). The main aspects of learner experience being examined in the included studies can be divided into three groups, namely: (1) Perceived presence, immersion and realism;…”

“…Of the 21 included studies 19 use exclusively or primarily quantitative methods. The remaining two are primarily qualitative (Fernandes et al, 2016;Reiners et al, 2014). To assess the quality of the 19 quantitative studies the Medical Education Research Study Quality Instrument (MERSQI) was employed (Reed et al, 2007).…”

“…There were two primarily qualitative studies, which could not be included in the quantitative MERSQI analysis. In Fernandes et al (2016) the researchers described six experiments each of which included two sources of data: observations of learners playing a VR based educational game and a questionnaire that the learners filled out after the experience. Their methodology follows an iterative design where elements, such as the questionnaire or the game itself, are adjusted and improved between the experiments.…”

“…Pan et al (2016) found that shortcomings in the visual presentation of virtual patients, such as lagging graphics, was limiting the sense of presence felt by medical professionals during the VR experience. When evaluating an educational video game, Fernandes et al (2016) found that the awareness of people watching you while you were wearing the HMD was limiting the sense of presence.…”