Analysis of natural gestures for controlling robot teams on multi-touch tabletop surfaces

Abstract

Multi-touch technologies hold much promise for the command and control of mobile robot teams. To improve the ease of learning and usability of these interfaces, we conducted an experiment to determine the gestures that people would naturally use, rather than the gestures they would be instructed to use in a pre-designed system. A set of 26 tasks with differing control needs were presented sequentially on a DiamondTouch to 31 participants. We found that the task of controlling robots exposed unique gesture sets and considerations not previously observed, particularly in desktop-like applications. In this paper, we present the details of these findings, a taxonomy of the gesture set, and guidelines for designing gesture sets for robot control.

Exploring the design space of gestural interaction with active tokens through user-defined gestures

“…Such studies have been used to design gesture interfaces of various types including multitouch gestures on small and large surfaces [2,10,11,22,25,32,57] and multi-modal interactions [28]. There is evidence that user-defined gesture sets are more complete than those defined solely by experts [2,32,35,57].…”

Hand occlusion on a multi-touch tabletop

“…We consulted multi-touch device SDKs and user guides (nicely summarized in [31]) and related research [3,5,7,11,16,17,30]. To keep the study reasonable and more ecologically valuable, we selected only gestures which use a single posture (e.g.…”

“Oh Snap” – Helping Users Align Digital Objects on Touch Interfaces

“…Wobbrock et al [33] investigated user-defined gestures for general interactions on multi-touch tabletops and found that touchingand-dragging is the most natural method for translating digital objects, and that dragging by the corner is the most natural way to rotate objects. Similarly, Micire et al [19] conducted an analysis of user-defined gestures for robot manipulation on a multi-touch tabletop. They too found that touch dragging was the most used gesture for positioning robots.…”

“…As a result, many researchers are investigating their usefulness for completing an increasingly diverse collection of tasks, including: the control of robots [19], the control of systems [8,21], managing artifacts [2,3,13], and software engineering [9]. Most of these systems support the selection and manipulation of digital objects on the screen using direct touch, exploiting the naturalness of physical direct interaction.…”