Gestu-Wan - An Intelligible Mid-Air Gesture Guidance System for Walk-up-and-Use Displays

Abstract

We present Gestu-Wan, an intelligible gesture guidance system designed to support mid-air gesture-based interaction for walk-up-and-use displays. Although gesture-based interfaces have become more prevalent, there is currently very little uniformity with regard to gesture sets and the way gestures can be executed. This leads to confusion, bad user experiences and users who rather avoid than engage in interaction using mid-air gesturing. Our approach improves the visibility of gesture-based interfaces and facilitates execution of mid-air gestures without prior training. We compare Gestu-Wan with a static gesture guide, which shows that it can help users with both performing complex gestures as well as understanding how the gesture recognizer works.

Designing guiding systems for gesture-based interaction

“…An ink trail provides a feedback by showing the complete path of the gesture [3]. Contrastingly, Gestu-Wan displays a partial portion by showing only the last 3D posture recognized during the recognition process [24].…”

Fortunettes

“…In gestural interfaces, for example, the gestures are often hard to discover and learn. To accommodate in-situ learning, OctoPocus [9] and Gestu-Wan [49] dynamically visualize the possible paths a user can follow to complete specific gestures. Similarly, TouchGhosts [54] facilitate discovering and learning multi-touch interactions by visualization not only the necessary actions, but also the effects of those actions on the current system state.…”

Exploring in-the-Wild Game-Based Gesture Data Collection

“…1) to collect an in-the-wild gesture corpus from a large and potentially diverse user population. The proposed system is based on a rhythm game combined with a Walk-Up-and-Use Display [4,5]. The system consists of a sensor for detecting 3D hand gestures and a computer with a large display for running a game.…”

“…The system can be deployed in a public space and store gestures to an online database, and the collected data may then be used to make a large gesture corpus that would help development and evaluation of robust gesture recognition algorithms. In such systems, however, users of the system should be informed of how to use such a display as they approach it, while also being attracted to use the system, which can be a challenging task for UI designers [4]. For in-the-wild settings, the systems should a) teach users how to interact with the system and b) help sustaining user engagement at least of one full game session.…”