Deixis: How to Determine Demonstrated Objects Using a Pointing Cone

Kranstedt, Alfred; Lücking, Andy; Pfeiffer, Thies; Rieser, Hannes; Wachsmuth, Ipke

doi:10.1007/11678816_34

Cited by 24 publications

(28 citation statements)

References 14 publications

(13 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Kranstedt et.al. [8] substantiated this model of human interpretation of pointing when presenting their pointing cone. Thus cone-based models, such as flashlight or aperture-based selection [7], have a broader applicability beyond 3D user interfaces.…”

Section: Ray Casting Vs Flashlightsupporting

confidence: 54%

Towards a linguistically motivated model for selection in virtual reality

Pfeiffer

2012

2012 IEEE Virtual Reality (VR)

Self Cite

View full text Add to dashboard Cite

Swiftness and robustness of natural communication is tied to the redundancy and complementarity found in our multimodal communication. Swiftness and robustness of human-computer interaction (HCI) is also a key to the success of a virtual reality (VR) environment. The interpretation of multimodal interaction signals has therefore been considered a high goal in VR research, e.g. following the visions of Bolt's put-that-there in 1980 [1].It is our impression that research on user interfaces for VR systems has been focused primarily on finding and evaluating technical solutions and thus followed a technology-oriented approach to HCI. In this article, we argue to complement this by a human-oriented approach based on the observation of human-human interaction. The aim is to find models of human-human interaction that can be used to create user interfaces that feel natural. As the field of Linguistics is dedicated to the observation and modeling of human-human communication, it could be worthwhile to approach natural user interfaces from a linguistic perspective.We expect at least two benefits from following this approach. First, the human-oriented approach substantiates our understanding of natural human interactions. Second, it brings about a new perspective by taking the interaction capabilities of a human addressee into account, which are not often explicitly considered or compared with that of the system. As a consequence of following both approaches to create user interfaces, we expect more general models of human interaction to emerge.

show abstract

Section: Ray Casting Vs Flashlightsupporting

confidence: 54%

Towards a linguistically motivated model for selection in virtual reality

Pfeiffer

2012

2012 IEEE Virtual Reality (VR)

Self Cite

View full text Add to dashboard Cite

show abstract

“…It achieves a good approximation when the pointing arm is extruded, but is less suitable for finger pointing or pointing with a bent arm. In fact, pointing is inherently inaccurate, as discussed in [43]. The authors estimated that a human pointing gesture has an angular uncertainty of approx.…”

Section: Related Workmentioning

confidence: 99%

“…This equation represents the probability p G (x) that a point x in the image plane was referred-to by the pointing gesture given the current head-shoulder detection d and the pointing directionô, and thus defines our corridor of attention. To account for the findings in [43], we set a lower bound of 3…”

Section: Realizationmentioning

confidence: 99%

Saliency-based identification and recognition of pointed-at objects

Schauerte

Richarz

Fink

2010

2010 IEEE/RSJ International Conference on Intelligent Robots and Systems

View full text Add to dashboard Cite

Abstract-When persons interact, non-verbal cues are used to direct the attention of persons towards objects of interest. Achieving joint attention this way is an important aspect of natural communication. Most importantly, it allows to couple verbal descriptions with the visual appearance of objects, if the referred-to object is non-verbally indicated. In this contribution, we present a system that utilizes bottom-up saliency and pointing gestures to efficiently identify pointed-at objects. Furthermore, the system focuses the visual attention by steering a pan-tilt-zoom camera towards the object of interest and thus provides a suitable model-view for SIFT-based recognition and learning. We demonstrate the practical applicability of the proposed system through experimental evaluation in different environments with multiple pointers and objects.

show abstract

“…Obviously, the accuracy of their measurement method was not very high. In a joint project, we replicated their original study [5], this time using a marker-based tracking system of our VR installation to track the exact positions of the index finger during the pointing tasks. The high accuracy and the high number of observed pointing gestures allowed a data-driven quantitative approach, which finally led to a precise model of pointing direction and extension [9].…”

Section: Recording Multimodal Interactionsmentioning

confidence: 99%

Using virtual reality technology in linguistic research

Pfeiffer

2012

2012 IEEE Virtual Reality (VR)

Self Cite

View full text Add to dashboard Cite

In this paper, we argue that empirical research on genuine linguistic topics, such as on the production of multimodal utterances in the speaker and the interpretation of the multimodal signals in the interlocutor, can greatly benefit from the use of virtual reality technologies.Established methodologies for research on multimodal interactions, like the presentation of pre-recorded 2D videos of interaction partners as stimuli and the recording of interaction partners using multiple 2D video cameras have crucial shortcomings regarding ecological validity and the precision of measurements that can be achieved. In addition, these methodologies enforce restrictions on the researcher. The stimuli, for example, are not very interactive and thus not as close to natural interactions as ultimately desired. Also, the analysis of 2D video recordings requires intensive manual annotations, often frame-by-frame, which negatively affects the feasible number of interactions which can be included in a study.The technologies bundled under the term virtual reality offer exciting possibilities for the linguistic researcher: gestures can be tracked without being restricted to fixed perspectives, annotation can be done on large corpora (semi-)automatically and virtual characters can be used to produce specific linguistic stimuli in a repetitive but interactive fashion. Moreover, immersive 3D visualizations can be used to recreate a simulation of the recorded interactions by fusing the raw data with theoretic models to support an iterative data-driven development of linguistic theories. This paper discusses the potential of virtual reality technologies for linguistic research and provides examples for the application of the methodology.

show abstract

Deixis: How to Determine Demonstrated Objects Using a Pointing Cone

Cited by 24 publications

References 14 publications

Towards a linguistically motivated model for selection in virtual reality

Towards a linguistically motivated model for selection in virtual reality

Saliency-based identification and recognition of pointed-at objects

Using virtual reality technology in linguistic research

Contact Info

Product

Resources

About