SenseShapes: using statistical geometry for object selection in a multimodal augmented reality

Olwal, Alex; Benko, Hrvoje; Feiner, Steven

doi:10.1109/ismar.2003.1240730

Cited by 53 publications

(41 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SenseShapes [14] aimed to find spatial correlation between gestures and deictic terms such as "that", "here", and "there" in an object selection task. The user's hands were tracked using data gloves, and object selection was facilitated by a virtual cone projected out from the users' fingers.…”

Section: Hand Gesture and Speech Interfaces In Armentioning

confidence: 99%

User-Defined Gestures for Augmented Reality

Piumsomboon

Clark

Billinghurst

et al. 2013

Human-Computer Interaction – INTERACT 2013

143

146

View full text Add to dashboard Cite

Abstract.Recently there has been an increase in research towards using hand gestures for interaction in the field of Augmented Reality (AR). These works have primarily focused on researcher designed gestures, while little is known about user preference and behavior for gestures in AR. In this paper, we present our guessability study for hand gestures in AR in which 800 gestures were elicited for 40 selected tasks from 20 participants. Using the agreement found among gestures, a user-defined gesture set was created to guide designers to achieve consistent user-centered gestures in AR. Wobbrock's surface taxonomy has been extended to cover dimensionalities in AR and with it, characteristics of collected gestures have been derived. Common motifs which arose from the empirical findings were applied to obtain a better understanding of users' thought and behavior. This work aims to lead to consistent user-centered designed gestures in AR.

show abstract

Section: Hand Gesture and Speech Interfaces In Armentioning

confidence: 99%

User-Defined Gestures for Augmented Reality

Piumsomboon

Clark

Billinghurst

et al. 2013

Human-Computer Interaction – INTERACT 2013

143

146

View full text Add to dashboard Cite

show abstract

“…However, in our AR applications we wanted to support natural 3D object interaction. Previously other researchers have used speech input for descriptive commands and used hand tracking devices or DataGloves [11][12] [13] to explore gesture input in 3D graphics environments. Alternatively, computer vision based hand tracking techniques have been used in systems such as "VisSpace" [14] to estimate where users were pointing.…”

Section: Related Workmentioning

confidence: 99%

“…One of the first multimodal AR interfaces, SenseShapes [11], used volumetric regions of interest that were attached to the user's gaze direction or hand to provide visual information about interaction with virtual objects. Object selection was available with a data glove to detect user's gestures and with trackers to monitor hand position for interaction with objects.…”

Section: Related Workmentioning

confidence: 99%

A Wizard of Oz study for an AR multimodal interface

Lee

Billinghurst

2008

Proceedings of the 10th International Conference on Multimodal Interfaces

View full text Add to dashboard Cite

In this paper we describe a Wizard of Oz (WOz) user study of an Augmented Reality (AR) interface that uses multimodal input (MMI) with natural hand interaction and speech commands. Our goal is to use a WOz study to help guide the creation of a multimodal AR interface which is most natural to the user. In this study we used three virtual object arranging tasks with two different display types (a head mounted display, and a desktop monitor) to see how users used multimodal commands, and how different AR display conditions affect those commands. The results provided valuable insights into how people naturally interact in a multimodal AR scene assembly task. For example, we discovered the optimal time frame for fusing speech and gesture commands into a single command. We also found that display type did not produce a significant difference in the type of commands used. Using these results, we present design recommendations for multimodal interaction in AR environments.

show abstract

“…Tabletop interaction (Section 4.1) allows 2D navigation of all objects and their relations to the various layers of the site. Used in conjunction with the [20,25] in conjunction with our VirtualTray (Section 4.5).…”

Section: Interaction In Vitamentioning

confidence: 99%

“…Object selection is the most frequently used 3D interaction technique, and we provide several ways to accomplish it. The user can walk toward an object and grab it, or point at it in the distance using our SenseShapes selection tools [25]. Selection at a distance is prone to ambiguity, especially when many similar objects are near each other, and SenseShapes assist in their disambiguation.…”

Section: D Multimodal Interactionmentioning

confidence: 99%