User-defined gestures for augmented reality

Piumsomboon, Thammathip; Clark, Adrian; Billinghurst, Mark; Cockburn, Andy

doi:10.1145/2468356.2468527

Cited by 170 publications

(40 citation statements)

References 15 publications

(26 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, relevant contributions from other countries can be found. For example, from Russia [133], Canada [20,[134][135][136], Taiwan [137,138], Japan [139][140][141][142][143], New Zealand [15,144,145], Switzerland [146][147][148][149], Singapore [150,151], China [31,152], Australia [35,[153][154][155][156][157][158][159], Korea [33,[160][161][162], Turkey [163,164], Egypt [165], Malaysia [6], or Israel [166,167].…”

Section: Discussionmentioning

confidence: 99%

Augmented and Virtual Reality Evolution and Future Tendency

2020

View full text Add to dashboard Cite

Augmented reality and virtual reality technologies are increasing in popularity. Augmented reality has thrived to date mainly on mobile applications, with games like Pokémon Go or the new Google Maps utility as some of its ambassadors. On the other hand, virtual reality has been popularized mainly thanks to the videogame industry and cheaper devices. However, what was initially a failure in the industrial field is resurfacing in recent years thanks to the technological improvements in devices and processing hardware. In this work, an in-depth study of the different fields in which augmented and virtual reality have been used has been carried out. This study focuses on conducting a thorough scoping review focused on these new technologies, where the evolution of each of them during the last years in the most important categories and in the countries most involved in these technologies will be analyzed. Finally, we will analyze the future trend of these technologies and the areas in which it is necessary to investigate to further integrate these technologies into society.

show abstract

Section: Discussionmentioning

confidence: 99%

Augmented and Virtual Reality Evolution and Future Tendency

2020

View full text Add to dashboard Cite

show abstract

“…'Air tap' was left out because it is similar to 'tap and hold' in every aspect except for its duration ('air tap' being more brief and thus less suitable for mid-air haptic actuation). In addition to the default HoloLens commands, we turned to Piumsomboon et al's user-defined AR gesture set [6] to supplement the HoloLens gestures. From it, we selected three additional gestures for our study: a) 'swipe' (used to rotate a carousel); b) 'stop' (used to pause, e.g., a song or video); and c) 'spin [index finger]' (used to increase or decrease the volume).…”

Section: Apparatus and Gesture Selectionmentioning

confidence: 99%

“…In principle, a symbol can be anything that computing systems are able to register as input. This method has been used to design symbols (input) to actuate, among others, touchscreens [13], virtual & augmented reality [6], smart glasses [9], home entertainment systems [10] and public displays [7]. Whereas up until now participants of end-user elicitation studies were tasked with the ideation of the symbol, we assigned ours with the elicitation of what we suggest to call intermediary referents.…”

Section: Introductionmentioning

confidence: 99%

User-Defined Mid-Air Haptic Sensations for Interacting with an AR Menu Environment

Bogaert

Geerts

2020

Haptics: Science, Technology, Applications

View full text Add to dashboard Cite

Interfaces that allow users to interact with a computing system by using free-hand mid-air gestures are becoming increasingly prevalent. A typical shortcoming of such gesture-based interfaces, however, is their lack of a haptic component. One technology with the potential to address this issue, is ultrasound mid-air haptic feedback. At the moment, haptic sensations are typically designed by system engineers and experts. In the case of gestural interfaces, researchers started involving non-expert users to define suitable gestures for specific interactions. To our knowledge, no studies have involved-from a similar participatory design perspective-laymen to generate mid-air haptic sensations. We present the results of an end-user elicitation study yielding a user-defined set of mid-air haptic sensations to match gestures used for interacting with an Augmented Reality menu environment. In addition, we discuss the suitability of the end-user elicitation method to that end.

show abstract

“…[ 25 , 27 ] came up with a taxonomy to describe and classify their gestures and measure the consensus according to those properties. Other user consensus works are not limited to the “2D” manipulation of the space, in [ 24 ] an elicitation study for augmented reality tools was conducted, finding a 29% agreement between subjects.…”

Section: Introductionmentioning

confidence: 99%

Gestures for Picture Archiving and Communication Systems (PACS) operation in the operating room: Is there any standard?

et al. 2018

View full text Add to dashboard Cite

ObjectiveGestural interfaces allow accessing and manipulating Electronic Medical Records (EMR) in hospitals while keeping a complete sterile environment. Particularly, in the Operating Room (OR), these interfaces enable surgeons to browse Picture Archiving and Communication System (PACS) without the need of delegating functions to the surgical staff. Existing gesture based medical interfaces rely on a suboptimal and an arbitrary small set of gestures that are mapped to a few commands available in PACS software. The objective of this work is to discuss a method to determine the most suitable set of gestures based on surgeon’s acceptability. To achieve this goal, the paper introduces two key innovations: (a) a novel methodology to incorporate gestures’ semantic properties into the agreement analysis, and (b) a new agreement metric to determine the most suitable gesture set for a PACS.Materials and methodsThree neurosurgical diagnostic tasks were conducted by nine neurosurgeons. The set of commands and gesture lexicons were determined using a Wizard of Oz paradigm. The gestures were decomposed into a set of 55 semantic properties based on the motion trajectory, orientation and pose of the surgeons’ hands and their ground truth values were manually annotated. Finally, a new agreement metric was developed, using the known Jaccard similarity to measure consensus between users over a gesture set.ResultsA set of 34 PACS commands were found to be a sufficient number of actions for PACS manipulation. In addition, it was found that there is a level of agreement of 0.29 among the surgeons over the gestures found. Two statistical tests including paired t-test and Mann Whitney Wilcoxon test were conducted between the proposed metric and the traditional agreement metric. It was found that the agreement values computed using the former metric are significantly higher (p < 0.001) for both tests.ConclusionsThis study reveals that the level of agreement among surgeons over the best gestures for PACS operation is higher than the previously reported metric (0.29 vs 0.13). This observation is based on the fact that the agreement focuses on main features of the gestures rather than the gestures themselves. The level of agreement is not very high, yet indicates a majority preference, and is better than using gestures based on authoritarian or arbitrary approaches. The methods described in this paper provide a guiding framework for the design of future gesture based PACS systems for the OR.

show abstract

User-defined gestures for augmented reality

Cited by 170 publications

References 15 publications

Augmented and Virtual Reality Evolution and Future Tendency

Augmented and Virtual Reality Evolution and Future Tendency

User-Defined Mid-Air Haptic Sensations for Interacting with an AR Menu Environment

Gestures for Picture Archiving and Communication Systems (PACS) operation in the operating room: Is there any standard?

Contact Info

Product

Resources

About