Action Affordance Affects Proximal and Distal Goal-oriented Planning

Abstract

Visual attention is mainly goal-directed and allocated based on the upcoming action to be performed. However, it is unclear how far this feature of gaze behavior generalizes in more naturalistic settings. The present study investigates active inference processes revealed by eye movements during interaction with familiar and novel tools with two levels of realism of the action affordance. In a between-subject design, a cohort of participants interacted with a VR controller in a low realism environment; another performed the task with an interaction setup that allowed differentiated hand and finger movements in a high-realism environment. We investigated the differences in odds of fixations and their eccentricity towards the tool parts before action initiation. The results show that participants fixate more on the tool’s effector part before action initiation when asked to use the tool and during interaction with unfamiliar tools. The spatial viewing bias on the tool reveals early fixations are influenced by the task and the familiarity of the tools. Our findings suggest that fixations are made in a task-oriented way to plan the intended action well before action initiation. With more realistic action affordances, more fixations were allocated toward the tool handle. We hypothesize that these fixations are made towards the proximal goal of planning the grasp even though the perceived action on the tools is identical for both experimental setups. Taken together, proximal and distal goal-oriented planning is contextualized to the realism of action/interaction afforded by an environment.

Combining EEG and Eye-Tracking in Virtual Reality - Obtaining Fixation-Onset ERPs and ERSPs

“…Using our eye movement classification algorithm, we showed that we could accurately classify eye movements of three-dimensional free-exploration data and that we can generate fERPs and fERSPs, proving that combining EEG and free-viewing virtual reality setups is possible. We investigated the classification quality using our modified version of a velocity-based classification algorithm (Dar et al, 2021;Keshava et al, 2023;Voloh et al, 2020), correcting for subject movement in the virtual environment. Furthermore, we compared two data-segmentation methods dealing with varying noise levels across a long recording.…”

“…There are a number of tools available to classify eye movements for static 2D images (e.g., Dar et al, 2021); however, we found that dynamic scenes created by the 3D environment, as well as allowing subjects to move, caused a problem. So, step by step, we modified and appended existing algorithms to this more demanding scenario (Dar et al, 2021;Keshava et al, 2023;Voloh et al, 2020). Specifically, adding EEG into the mix considerably increased the demand for precision (Luck, 2014).…”

“…We, therefore, introduced a movement correction before calculating the eye angular velocity (Keshava et al, 2023) used to define gazes and saccades. The key to our approach is first to compute the hit point's movement in allocentric coordinates and then translate it to the required change of eye direction in allocentric coordinates.…”

“…The calculated angular velocities (Keshava et al, 2023) were classified into gaze events (smooth pursuit and fixations) and saccades based on an adaptively calculated threshold (Voloh et al, 2020). This experiment did not have predefined trials, so applying the adaptive threshold method to the entire dataset would only detect very big saccades.…”

“…We created a virtual, three-dimensional environment for our participants to explore while recording eye-tracking and EEG. Eye movements were classified using a velocity-based algorithm and an adaptively determined velocity threshold (Keshava et al, 2023;Voloh et al, 2020).…”