The effect of the modality principle to support learning with virtual reality: An eye‐tracking and electrodermal activity study

Abstract

Background Virtual reality (VR) is considered a promising approach to support learning. An instructional design is essential to optimize cognitive processes. Studies show that VR has unique instructional and pedagogical requirements. Objectives To evaluate the effectiveness and applicability of the modality principle, which was previously validated in 2D classic multimedia, for learning with VR. The modality principle states that multimedia information presented as spoken narration is superior to on‐screen text. Methods A prospective experimental study with two compared conditions of instruction: VR‐based learning guided by on‐screen text (n = 34) versus spoken narration (n = 28). Students' cognitive learning experiences were captured by eye‐tracking and electrodermal activity (EDA). In addition, students' knowledge was evaluated using a pre–post knowledge test. Results and Conclusions Overall, there was no significant difference in knowledge retention between the participants who learned with on‐screen text compared to spoken narration. However, results from the eye‐tracking analysis showed that students who learned with the on‐screen text devoted longer visual attention toward important learning activity areas of interest, suggesting a better ability to discern between relevant and irrelevant information. Conversely, students who learned with the spoken narration expressed significantly more EDA peak responses, proposing a higher cognitive load. Implications This study outlines that while learning with VR was effective, the modality principle might not apply to learning with VR. Moreover, the analysis of the learning process suggests even an inverse effect, favouring the provision of instructional scaffolds as on‐screen text. Future research should evaluate this effect on long‐term knowledge retention.

Realistic details impact learners independently of split-attention effects

“…As discussed above, results from studies on learning with visualizations do not necessarily provide evidence that an effect needs to be reproducible when learning in virtual reality (Liberman and Dubovi 2022 ). Thus, we need to be careful when attempting to transfer the results of the present studies to learning in virtual worlds.…”

“…Interestingly, a recent study found that instructional principles that have been well-established within other media do not necessarily need to be transferable to learning in virtual reality. Liberman and Dubovi ( 2022 ) established that simultaneously using multiple sensory modalities, in their case the visual and auditory senses to present text either as narration or on-screen, does not improve learning in virtual reality despite positive results of this approach for other types of media (Moreno and Mayer 1999 ). Thus, the remainder of this overview will be focused on learning with visualizations rather than with entire virtual environments.…”

Concept of Virtual Reality Training of Facility Management Employees Using Eye Tracking

“…An alternative to VR high-immersive training using a headset is low-immersive VR training, often called desktop VR. The low-immersive VR training is a cheaper alternative that only needs to display the virtual world on a computer monitor [31]. However, it does not ensure the measurement of a high range of parameters available in high-immersive training.…”