The Semantic Meaning of Hand Shapes and Z-Dimension Movements of Freehand Distal Pointing on Large Displays

Abstract

It is difficult to select small targets in freehand distal pointing on large displays due to physical fatigue and jitter. Previous research proposed many solutions but not through the semantic meaning of hand behaviors. The purpose of this study is to investigate the hand behaviors, including hand shapes and motions in the Z-dimension between small and large target selections, by means of two controlled experiments. The generated results indicated that the following: (1) when interacting with a large display from a remote distance, users preferred to complete a low-precision task by the whole hand while using the finger in high-precision tasks. As a result, users moved small targets by the index finger gesture while moving large targets by open hand gesture; (2) selection of small targets led to hand stretching which might reduce the interaction distance; (3) the orientation had significant effects on hand motions in the Z-dimension. The relationship between orientation and hand motions in the Z-dimension is similar to a sinusoidal function; (4) mouse-like pointing had much impact on freehand interaction. Based on these results, some design guidelines for freehand interaction and design suggestions for freehand pointing were discussed.

Mapping Directional Mid-Air Unistroke Gestures to Interaction Commands: A User Elicitation and Evaluation Study

“…As the basic body movements that humans naturally and repetitiously perform, stroke gestures are of a motor-intuitive nature [14] and so have been widely applied in touch-based interfaces for pen input [15][16][17], text-editing [18] smartphone use [19] and game [20]. Stroke gesture is also a common input for mid-air interaction, in particular for the control of cursor position [21] and distal pointing [22] on distant displays [23]. There are a number of studies focusing on the in-air limb movement for completing interactive tasks related to manipulating a 3D object [24], ubiquitous environment [25] and TV control [3].…”