An investigation of feed-forward and feed-back eye movement training in immersive virtual reality

Abstract

The control of eye gaze is critical to the execution of many skills. The observation that task experts in many domains exhibit more efficient control of eye gaze than novices has led to the development of gaze training interventions that teach these behaviours. We aimed to extend this literature by i) examining the relative benefits of feed-forward (observing an expert’s eye movements) versus feed-back (observing your own eye movements) training, and ii) automating this training within virtual reality. Serving personnel from the British Army and Royal Navy were randomised to either feed-forward or feed-back training within a virtual reality simulation of a room search and clearance task. Eye movement metrics – including visual search, saccade direction, and entropy – were recorded to quantify the efficiency of visual search behaviours. Feed-forward and feed-back eye movement training produced distinct learning benefits, but both accelerated the development of efficient gaze behaviours. However, we found no evidence that these more efficient search behaviours transferred to better decision making in the room clearance task. Our results suggest integrating eye movement training principles within virtual reality training simulations may be effective, but further work is needed to understand the learning mechanisms.

Attention computing for enhanced visuomotor skill performance: Testing the effectiveness of gaze-adaptive cues in virtual reality golf putting

“…In the context of golfing skills, for example, intelligent monitoring systems could be designed to improve an individual's pre-performance routine, enhance the precise 'coupling' of their hand and eye movements, or even to support their reading of a course (e.g., by cueing eye-gaze towards significant topographical features that had not been attended to). Drawing on previous work in aviation [24,25,61], expert gaze patterns could then be used as a feedforward learning cue, while automated data algorithms could be used to index complex (and often undetected) underlying states (e.g., changes in anxiety, cognitive load, or task expertise). Hence the first steps demonstrated here illustrate the farreaching potential of these technology-enhanced training principles.…”

“…Immersive technologies, like virtual reality (VR), augmented reality, (AR) and mixed reality (XR) are becoming popular mediums for delivering training in industries like defence [24,25], surgery [26], rehabilitation [39], and sport [5,49,50]. VR is attractive for training providers because it delivers high levels of immersion [6,44], can increase motivation to train [44,57], and enables individuals to actively practice for complex and dangerous situations [46].…”

“…Instead of using immersive technologies simply as a replacement for existing training, there is also an opportunity to open up new pedagogical approaches. Indeed, VR allows the user to experience environments which are not physically possible in the 'real world', and can allow learners to get automated feedback on their performance by watching themselves from a third person perspective [24,25] or even to adopt the perspectives of other people [63,83]. One particularly promising avenue for improved pedagogy is adaptive VR.…”

“…Secondly, through showing the trainee the typical gaze patterns of domain experts, which is known as either eye movement modelling [31,72] or feedforward eye movement training [61,80,85]. Finally, the trainee can be shown their own eye movements as a form of feedback, which is known as feed-back eye movement training [24,25,81].…”

Introduction to the Special Thematic Issue "Virtual Reality and Eye Tracking"

“…Overall, the articles of the present special thematic issue showcase a variety of use cases and applications of VR-based eye tracking in research and practice. They show how eye tracking in VR can be and has already been used to advance research in various fields, including research on the human visual system (Beitner et al, 2023;Prummer et al, 2024) and cognitive processes (Pettersson et al, 2018;Słowiński et al, 2022;Souchet et al, 2022), with important practical applications, for example in visual field examinations (Portengen et al, 2022) and assessment of visual acuity (Kim et al, 2016;, military training (Harris et al, 2023) and for the development of new technological systems (Hebbar et al, 2023). Nevertheless, these articles also indicate there are still many open technical and methodological questions (Freytag et al, 2023;Pielage et al, 2023;Schuetz & Fiehler, 2022) that need to be addressed in future research to establish the required standards for scientific practice and an empirical basis for valid and reliable applications in various fields.…”