Identifying the Usability Factors of Mid-Air Hand Gestures for 3D Virtual Model Manipulation

Abstract

Abstract. Although manipulating 3D virtual models with mid-air hand gestures had the benefits of natural interactions and free from the sanitation problems of touch surfaces, many factors could influence the usability of such an interaction paradigm. In this research, the authors conducted experiments to study the visionbased mid-air hand gestures for scaling, translating, and rotating a 3D virtual car displayed on a large screen. An Intel RealSense 3D Camera was employed for hand gesture recognition. The two-hand gesture with grabbing then moving apart/close to each other was applied to enlarging/shrinking the 3D virtual car. The one-hand gesture with grabbing then moving was applied to translating a car component. The two-hand gesture with grabbing and moving relatively along the circumference of a horizontal circle was applied to rotating the car. Seventeen graduate students were invited to participate in the experiments and offer their evaluations and comments for gesture usability. The results indicated that the width and depth of detection ranges were the key usability factors for two-hand gestures with linear motions. For dynamic gestures with quick transitions and motions from open to close hand poses, ensuring gesture recognition robustness was extremely important. Furthermore, given a gesture with ergonomic postures, inappropriate control-response ratio could result in fatigue due to repetitive exertions of hand gestures for achieving the precise controls of 3D model manipulation tasks.

Child–display interaction: Lessons learned on touchless avatar-based large display interfaces

“…In the context of pervasive displays research, many touchless gestural interfaces have been proposed and implemented. They have been used in order to interact with 3D virtual objects [7], to access information provision systems [8,19], to create and support playful interactions [26], and in many other applications. The use of touchless gestures, especially when applied to public displays, has many advantages.…”

Drawing with AI -- Exploring Collaborative Inking Experiences Based on Mid-air Pointing and Reinforcement Learning

“…Previous work has extensively explored mid-air input as a modality for human-computer interaction and designed its use for contexts, such as cars [13], clean rooms in industrial settings [14], and for retail situations [15], but also for pervasive displays [5,6]. Aslan et al [16] showed that interfaces reacting to the movement of a user's hand before any touch event happens can appear more attractive to the user.…”

“…In order to explore this notion of interacting with an intelligent digital canvas, we developed a drawing application based on reinforcement learning. Since mid-air pointing seems to be an engaging, "natural", and flexible way to draw on pervasive displays (e.g., [5,6]) we decided to integrate it as an alternative interaction technique for users. Before we describe the design of the drawing application in detail and report on a qualitative user study with a heterogeneous group of users we provide the necessary background in reinforcement learning and related work in mid-air interaction in the next section.…”

Child-display interaction

“…In the context of pervasive displays research, many touchless gestural interfaces have been proposed and implemented. They have been used in order to interact with 3D virtual objects [6], to access information provision systems [7,18], to create and support playful interactions [23], and have many other applications. The use of touchless gestures, especially when applied to public displays, has many advantages.…”