Gesture Elicitation Studies for Mid-Air Interaction: A Review

Abstract

Mid-air interaction involves touchless manipulations of digital content or remote devices, based on sensor tracking of body movements and gestures. There are no established, universal gesture vocabularies for mid-air interactions with digital content or remote devices based on sensor tracking of body movements and gestures. On the contrary, it is widely acknowledged that the identification of appropriate gestures depends on the context of use, thus the identification of mid-air gestures is an important design decision. The method of gesture elicitation is increasingly applied by designers to help them identify appropriate gesture sets for mid-air applications. This paper presents a review of elicitation studies in mid-air interaction based on a selected set of 47 papers published within 2011-2018. It reports on: (1) the application domains of mid-air interactions examined; (2) the level of technological maturity of systems at hand; (3) the gesture elicitation procedure and its variations; (4) the appropriateness criteria for a gesture; (5) participants number and profile; (6) user evaluation methods (of the gesture vocabulary); (7) data analysis and related metrics. This paper confirms that the elicitation method has been applied extensively but with variability and some ambiguity and discusses under-explored research questions and potential improvements of related research.Multimodal Technologies and Interact. 2018, 2, 65 2 of 21 such thing as a universal gesture vocabulary for every application" [15]. Thus, the identification of appropriate gestures for mid-air user interactions in terms of criteria like discoverability, memorability, performance, reliability and comfort, is an important design decision. This must be made at the early stages of system development and it severely affects the development course of every mid-air application project as well as the user experience (UX) of intended users.Emerging from the field of participatory design, gesture elicitation studies have been widely applied to help designers select the most appropriate gesture set for a given application. Although the method does not have a strict procedure, the main approach is to first define what operations (called "referents") have to be executed through gestures, then ask the end users to propose at least one gesture that they find preferable for each referent and finally to extract the gesture vocabulary after analysing the collected data [16]. The goal of a gesture elicitation study is to extract "good" gestures, which according to Morris et al. [17] are "gestures that meet certain design criteria such as discoverability, ease-of-performance, memorability, or reliability."Over the last few years, several elicitation studies have been conducted to identify mid-air gesture vocabularies for many different application domains and contexts, devices, types of digital content and tracking technologies subsumed. A survey of elicitation studies of mid-air interaction is useful to HCI researchers and interaction designers for deepening th...

Frame-Based Elicitation of Mid-Air Gestures for a Smart Home Device Ecosystem

“…Several elicitation studies of mid-air gestures for control of home devices have been conducted. In a recent review of 47 mid-air gesture elicitation studies [18], mid-air interaction with the smart home is investigated by 8.5% of papers examined while there are additional studies about control of digital content and devices that can be used in the home, like the TV (12.8%) and various media (14.9%).…”

“…), since users might associate them with specific gestures. There are several approaches of referent presentation in elicitation studies [18]: GUI animations, text messages on screen, described verbally, still images, and illustrations of manipulations of the actual artifacts. Apparently, if a referent has been designed in detail, it would have included widgets and handles which present affordances to participants of a gesture elicitation study.…”

Approach for Intuitive and Touchless Interaction in the Operating Room

“…This type of methodology has been gaining traction in various fields of interaction research in the recent years. Notably, Vogiatzidakis et al [13] provided a survey over "methods of gestures elicitation" regarding touchless interaction across various domains of application (e.g., virtual reality, desktop, public display, etc.). It appears that the "operating room" was among the least represented with only one study over 47 selected papers, showing the partial lack of such method applications in a domain where the usefulness of touchless interaction and its potential have been demonstrated.…”

Individual Mid-Air Gesture Sets Informed by Conceptual Metaphors: A Case Study on How Users Generate Mid-Air Gesture Sets to Control Video Streaming

“…Metaphorical gestures rely on semantic associations between the gestures and delivered functions. Abstract gestures are the remaining gestures that do not convey any association with the referred function [8]. Often, based on this taxonomy, researchers employed end-user elicitation studies and guessability method to standardize mid-air gesture sets.…”