The Virtual Mitten: A novel interaction paradigm for visuo-haptic manipulation of objects using grip force

Abstract

Figure 1: Visuo-haptic manipulation as enabled by our novel approach called the "Virtual Mitten". Each hand holds an elastic device to control a corresponding virtual mitten (in gray) and to grasp virtual objects in a bimanual scenario. The grip force applied by the user is measured to generate pseudo-haptic feedback. ABSTRACTIn this paper, we propose a novel visuo-haptic interaction paradigm called the "Virtual Mitten" for simulating the 3D manipulation of objects. Our approach introduces an elastic handheld device that provides a passive haptic feedback through the fingers and a mitten interaction metaphor that enables to grasp and manipulate objects. The grasping performed by the mitten is directly correlated with the grip force applied on the elastic device and a supplementary pseudo-haptic feedback modulates the visual feedback of the interaction in order to simulate different haptic perceptions. The Virtual Mitten allows natural interaction and grants users with an extended freedom of movement compared with rigid devices with limited workspaces. Our approach has been evaluated within two experiments focusing both on subjective appreciation and perception. Our results show that participants were able to well perceive different levels of effort during basic manipulation tasks thanks to our pseudo-haptic approach. They could also rapidly appreciate how to achieve different actions with the Virtual Mitten such as opening a drawer or pulling a lever. Taken together, our results suggest that our novel interaction paradigm could be used in a wide range of applications involving one or two-hand haptic manipulation such as virtual prototyping, virtual training or video games.

Visual aids in 3D point selection experiments

“…Color change is a commonly used visual feedback mechanisms. The recent Virtual Mitten technique [1], for example, uses colour changes to indicate pressure applied to a handheld grip device. Other recent work focused on visual feedback for hand-based grasp techniques [29].…”

3D Virtual Hand Selection with EMS and Vibration Feedback

“…Virtual hand-based techniques rely on the 3D intersection of the finger/hand/3D cursor with a target and thus require that the user picks the correct distance, i.e., visual depth. In such techniques, color change is the most commonly used visual feedback mechanism [1,17].…”

“…One of two main approaches to 3D selection is virtual finger/hand/3D cursor-based techniques [1,3,16]. The other approach is ray-based.…”

Leveraging Passive Haptic Feedback in Virtual Environments with the Elastic-Arm Approach

“…This effect was illustrated with a task involving the insertion of an object into a duct: when the manipulated object reaches the duct, the movement is slowed down and users increase the force applied on the passive device which makes them perceive a friction force. Pseudo-haptic feedback can also simulate other 6 haptic properties such as mass (Dominjon, Lécuyer, Burkhardt, Richard, & Richir, 2005), torque (Paljic, Burkhardt, & Coquillart, 2004), and grasping forces (Achibet et al, 2014).…”