A handle bar metaphor for virtual object manipulation with mid-air interaction

Abstract

Commercial 3D scene acquisition systems such as the Microsoft Kinect sensor can reduce the cost barrier of realizing mid-air interaction. However, since it can only sense hand position but not hand orientation robustly, current mid-air interaction methods for 3D virtual object manipulation often require contextual and mode switching to perform translation, rotation, and scaling, thus preventing natural continuous gestural interactions. A novel handle bar metaphor is proposed as an effective visual control metaphor between the user's hand gestures and the corresponding virtual object manipulation operations. It mimics a familiar situation of handling objects that are skewered with a bimanual handle bar. The use of relative 3D motion of the two hands to design the mid-air interaction allows us to provide precise controllability despite the Kinect sensor's low image resolution. A comprehensive repertoire of 3D manipulation operations is proposed to manipulate single objects, perform fast constrained rotation, and pack/align multiple objects along a line. Three user studies were devised to demonstrate the efficacy and intuitiveness of the proposed interaction techniques on different virtual manipulation scenarios.

INSPECT: extending plane-casting for 6-DOF control

“…Song et al used a Kinect to track the user's limbs in front of a large display and proposed a handlebar metaphor [9] for 3D object manipulation. Their users, however, complained about fatigue.…”

“…No big arm/hand gestures An interface that is to be used on a daily basis and/or for many hours has to avoid large hand gestures which are bound to induce fatigue [9] and, in rare cases, even cause physical injuries to bystanders.…”

Gaze and gesture based object manipulation in virtual worlds

“…We combine a "look at" mechanism for choosing objects with a handle bar metaphor [2] for object manipulation. Hand state is used as a trigger for selection and deselection of objects while eye gaze data is used to determine the object on which to apply the selection.…”

“…Motivated by interfaces depicted in films such as Minority Report, much attention has been placed on the use of spatial gestures for interactions in 3D virtual spaces [1,2]. Research on natural gaze behavior and hand-eye coordination during object manipulation suggests that the use of gaze data, in addition to gesture based methods, may increase the effectiveness of performing these tasks in a virtual space.…”

An Evaluation on Two-Handed and One-Handed Control Methods for Positioning Object in Immersive Virtual Environments

“…A handle bar metaphor interaction technique [16] had been proposed as a mid-air interaction technique using a camera-based motion capture device, and the technique is smart and effective. Mid-air interaction has recently attracted more research interest because camera-based motion capture systems like Microsoft Kinect have been popular and widely used.…”