Control of aperture closure during reach-to-grasp movements in immersive haptic-free virtual reality

Abstract

Virtual reality (VR) has garnered much interest as a training environment for motor skill acquisition, including for neurological rehabilitation of upper extremities. While the focus has been on gross upper limb motion, VR applications that involve reaching for, and interacting with, virtual objects are growing. The absence of true haptics in VR when it comes to hand-object interactions raises a fundamentally important question: can haptic-free immersive virtual environments (hf-VEs) support naturalistic coordination of reach-to-grasp movements? This issue has been grossly understudied, and yet is of significant importance in the development and application of VR across a number of sectors. In a previous study (Furmanek et al. 2019), we reported that reach-to-grasp movements are similarly coordinated in both the physical environment (PE) and hf-VE. The most noteworthy difference was that the closure phase, which begins at maximum aperture and lasts through the end of the movement, was longer in hf-VE than in PE, suggesting that different control laws might govern the initiation of closure between the two environments. To do so, we reanalyzed data from Furmanek et al. (2019), in which the participants reached to grasp three differently sized physical objects, and matching 3D virtual object renderings, placed at three different locations. Our analysis revealed two key findings pertaining to the initiation of closure in PE and hf-VE. First, the respective control laws governing the initiation of aperture closure in PE and hf-VE both included state estimates of transport velocity and acceleration, supporting a general unified control scheme for implementing reach-to-grasp across physical and virtual environments. Second, aperture was less informative to the control law in hf-VE. We suggest that the latter was likely because transport velocity at closure onset and aperture at closure onset were less independent in hf-VE than in PE, ultimately resulting in aperture at closure onset having a weaker influence on the initiation of closure. In this way, the excess time and muscular effort needed to actively bring the fingers to a stop at the interface of a virtual object was factored into the control law governing the initiation of closure in hf-VE. Crucially, this control law remained applicable, albeit with different weights in hf-VE, despite the absence of terminal haptic feedback and potential perceptual differences.

Hand Tracking for Immersive Virtual Reality: Opportunities and Challenges

“…It is likely that the main reason that controllerfree hand tracking is problematic during object interaction is the lack of tactile and haptic cues in this context. Tactile cues are a key part to successful manual actions, and their removal impairs the accuracy of manual localization (Rao and Gordon, 2001), alters grasping kinematics (Furmanek et al, 2019;Mangalam et al, 2021;Ozana et al, 2020;Whitwell et al, 2015), and affects the normal application of fingertip forces (Buckingham et al, 2016). While controllerbased interactions with virtual objects do not deliver the same tactile and haptic sensations experienced when interacting with objects in the physical environment, the vibro-tactile pulses and the mass of the controllers do seem to aid in scaffolding a compelling percept of touching something.…”