Understanding 3D Mid-Air Hand Gestures with Interactive Surfaces and Displays: A Systematic Literature Review

Groenewald, Celeste; Anslow, Craig; Islam, Junayed; Rooney, Chris; Passmore, Peter J.; Wong, William

doi:10.14236/ewic/hci2016.43

Cited by 33 publications

(13 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Each participant completed the tasks for scaling the car, translating the car seat, and rotating the car with respect to the vertical axis using designated hand gestures as follows. Among the diverse gesture types, "grab and move" and bimanual "handle bar metaphor" were reported as the intuitive gestures for object manipulation tasks [1,2,25]. In addition, users often preferred using gestures resembling physical manipulation for wall-sized displays [26].…”

Section: Methodsmentioning

confidence: 99%

“…In many applications, such as interactive and virtual product exhibition in public spaces and medical image displays in surgery rooms, the tasks may include scaling, translation, and rotation of 3D components. In order to ensure the natural mapping between controls and displays, the selections of mid-air hand gestures for different tasks were based on the metaphors of physical object operations [1] or the gestures from user-elicitation studies [2]. Given a consensus hand gesture type for a specific task, the performance of gesture recognition and control could still be influenced by many factors, such as the moving speed and trajectory of hands, the occlusion of fingers due to hand pose changes and transitions, as well as the individual differences of performing a specific hand gesture.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Identifying the Usability Factors of Mid-Air Hand Gestures for 3D Virtual Model Manipulation

Chen

Cheng

Chu

et al. 2017

Universal Access in Human–Computer Interaction. Designing Novel Interactions

View full text Add to dashboard Cite

Abstract. Although manipulating 3D virtual models with mid-air hand gestures had the benefits of natural interactions and free from the sanitation problems of touch surfaces, many factors could influence the usability of such an interaction paradigm. In this research, the authors conducted experiments to study the visionbased mid-air hand gestures for scaling, translating, and rotating a 3D virtual car displayed on a large screen. An Intel RealSense 3D Camera was employed for hand gesture recognition. The two-hand gesture with grabbing then moving apart/close to each other was applied to enlarging/shrinking the 3D virtual car. The one-hand gesture with grabbing then moving was applied to translating a car component. The two-hand gesture with grabbing and moving relatively along the circumference of a horizontal circle was applied to rotating the car. Seventeen graduate students were invited to participate in the experiments and offer their evaluations and comments for gesture usability. The results indicated that the width and depth of detection ranges were the key usability factors for two-hand gestures with linear motions. For dynamic gestures with quick transitions and motions from open to close hand poses, ensuring gesture recognition robustness was extremely important. Furthermore, given a gesture with ergonomic postures, inappropriate control-response ratio could result in fatigue due to repetitive exertions of hand gestures for achieving the precise controls of 3D model manipulation tasks.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Identifying the Usability Factors of Mid-Air Hand Gestures for 3D Virtual Model Manipulation

Chen

Cheng

Chu

et al. 2017

Universal Access in Human–Computer Interaction. Designing Novel Interactions

View full text Add to dashboard Cite

show abstract

“…Inspired by Wobbrock et al's [23] seminal work, several researchers further explored user gestures for a variety of tasks and application scenarios. The review of about 65 papers in [58] crystallized the insight that most gestures so far have been designed for selection, navigation, and manipulation tasks.…”

Section: Gestural Interaction With Public Displaysmentioning

confidence: 99%

User-Generated Gestures for Voting and Commenting on Immersive Displays in Urban Planning

Degbelo

Kray

2019

MTI

View full text Add to dashboard Cite

Traditional methods of public consultation offer only limited interactivity with urban planning materials, leading to a restricted engagement of citizens. Public displays and immersive virtual environments have the potential to address this issue, enhance citizen engagement and improve the public consultation process, overall. In this paper, we investigate how people would interact with a large display showing urban planning content. We conducted an elicitation study with a large immersive display, where we asked participants (N = 28) to produce gestures to vote and comment on urban planning material. Our results suggest that the phone interaction modality may be more suitable than the hand interaction modality for voting and commenting on large interactive displays. Our findings may inform the design of interactions for large immersive displays, in particular, those showing urban planning content.

show abstract

“…This requires careful reviews and assessments of the content of design suggestions for particular interactive technologies or contexts of use. For example, in the work of [75], a systematic survey of mid-air hand gestures with interactive surfaces and displays is presented. More content-based surveys of this sort can help the community to summarize and reflect on previous results and provide feedthrough and inspiration for analogous design contexts.…”

Section: Variability Of Application Domains and Systems' Technologicamentioning

confidence: 99%

Gesture Elicitation Studies for Mid-Air Interaction: A Review

Vogiatzidakis

Koutsabasis

2018

MTI

View full text Add to dashboard Cite

Mid-air interaction involves touchless manipulations of digital content or remote devices, based on sensor tracking of body movements and gestures. There are no established, universal gesture vocabularies for mid-air interactions with digital content or remote devices based on sensor tracking of body movements and gestures. On the contrary, it is widely acknowledged that the identification of appropriate gestures depends on the context of use, thus the identification of mid-air gestures is an important design decision. The method of gesture elicitation is increasingly applied by designers to help them identify appropriate gesture sets for mid-air applications. This paper presents a review of elicitation studies in mid-air interaction based on a selected set of 47 papers published within 2011-2018. It reports on: (1) the application domains of mid-air interactions examined; (2) the level of technological maturity of systems at hand; (3) the gesture elicitation procedure and its variations; (4) the appropriateness criteria for a gesture; (5) participants number and profile; (6) user evaluation methods (of the gesture vocabulary); (7) data analysis and related metrics. This paper confirms that the elicitation method has been applied extensively but with variability and some ambiguity and discusses under-explored research questions and potential improvements of related research.Multimodal Technologies and Interact. 2018, 2, 65 2 of 21 such thing as a universal gesture vocabulary for every application" [15]. Thus, the identification of appropriate gestures for mid-air user interactions in terms of criteria like discoverability, memorability, performance, reliability and comfort, is an important design decision. This must be made at the early stages of system development and it severely affects the development course of every mid-air application project as well as the user experience (UX) of intended users.Emerging from the field of participatory design, gesture elicitation studies have been widely applied to help designers select the most appropriate gesture set for a given application. Although the method does not have a strict procedure, the main approach is to first define what operations (called "referents") have to be executed through gestures, then ask the end users to propose at least one gesture that they find preferable for each referent and finally to extract the gesture vocabulary after analysing the collected data [16]. The goal of a gesture elicitation study is to extract "good" gestures, which according to Morris et al. [17] are "gestures that meet certain design criteria such as discoverability, ease-of-performance, memorability, or reliability."Over the last few years, several elicitation studies have been conducted to identify mid-air gesture vocabularies for many different application domains and contexts, devices, types of digital content and tracking technologies subsumed. A survey of elicitation studies of mid-air interaction is useful to HCI researchers and interaction designers for deepening th...

show abstract

Understanding 3D Mid-Air Hand Gestures with Interactive Surfaces and Displays: A Systematic Literature Review

Cited by 33 publications

References 25 publications

Identifying the Usability Factors of Mid-Air Hand Gestures for 3D Virtual Model Manipulation

Identifying the Usability Factors of Mid-Air Hand Gestures for 3D Virtual Model Manipulation

User-Generated Gestures for Voting and Commenting on Immersive Displays in Urban Planning

Gesture Elicitation Studies for Mid-Air Interaction: A Review

Contact Info

Product

Resources

About