Fast and Precise Touch-Based Text Entry for Head-Mounted Augmented Reality with Variable Occlusion

Dudley, John J.; Vertanen, Keith; Kristensson, Per Ola

doi:10.1145/3232163

Cited by 50 publications

(27 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Participants achieved 29 wpm after one hour of practice although stimulus phrases were selected to ensure only words in the known vocabulary were included. VISAR [4] also leverages probabilistic decoding, in an approach derived from Vertanen et al [31], and demonstrates single-finger mid-air text input specifically tailored for AR HMDs. After various refinements, including the provision of error-tolerant word predictions, the touch-based approach yielded a mean entry rate of 17.8 wpm.…”

Section: Related Workmentioning

confidence: 99%

Performance Envelopes of Virtual Keyboard Text Input Strategies in Virtual Reality

Dudley

Benko

Wigdor

et al. 2019

2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)

Self Cite

View full text Add to dashboard Cite

Virtual and Augmented Reality deliver engaging interaction experiences that can transport and extend the capabilities of the user. To ensure these paradigms are more broadly usable and effective, however, it is necessary to also deliver many of the conventional functions of a smartphone or personal computer. It remains unclear how conventional input tasks, such as text entry, can best be translated into virtual and augmented reality. In this paper we examine the performance potential of four alternative text entry strategies in virtual reality (VR). These four strategies are selected to provide full coverage of two fundamental design dimensions: i) physical surface association; and ii) number of engaged fingers. Specifically, we examine typing with index fingers on a surface and in mid-air and typing using all ten fingers on a surface and in mid-air. The central objective is to evaluate the human performance potential of these four typing strategies without being constrained by current tracking and statistical text decoding limitations. To this end we introduce an auto-correction simulator that uses knowledge of the stimulus to emulate statistical text decoding within constrained experimental parameters and use high-precision motion tracking hardware to visualise and detect fingertip interactions. We find that alignment of the virtual keyboard with a physical surface delivers significantly faster entry rates over a mid-air keyboard. Also, users overwhelmingly fail to effectively engage all ten fingers in mid-air typing, resulting in slower entry rates and higher error rates compared to just using two index fingers. In addition to identifying the envelopes of human performance for the four strategies investigated, we also provide a detailed analysis of the underlying features that distinguish each strategy in terms of its performance and behaviour.

show abstract

Section: Related Workmentioning

confidence: 99%

Performance Envelopes of Virtual Keyboard Text Input Strategies in Virtual Reality

Dudley

Benko

Wigdor

et al. 2019

2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Gesture-based interaction tracks the position and orientation of the user's fingers or hands using the camera or inertial measurement unit (IMU) of the AR smart glasses and supports interactions with virtual objects in the AR environments [21][22][23][24][25][26][27][28][29][30]. Ha et al [21] proposed WeARHand, which allows the user to manipulate virtual 3D objects with a bare hand in a wearable AR environment.…”

Section: Gesture Interactionsmentioning

confidence: 99%

User Interactions for Augmented Reality Smart Glasses: A Comparative Evaluation of Visual Contexts and Interaction Gestures

et al. 2019

View full text Add to dashboard Cite

Smart glasses for wearable augmented reality (AR) are widely used in various applications, such as training and task assistance. However, as the field of view (FOV) in the current AR smart glasses is narrow, it is difficult to visualize all the information on the AR display. Besides, only simple interactions are supported. This paper presents a comparative and substantial evaluation of user interactions for wearable AR concerning visual contexts and gesture interactions using AR smart glasses. Based on the evaluation, it suggests new guidelines for visual augmentation focused on task assistance. Three different types of visual contexts for wearable AR were implemented and evaluated: stereo rendering and direct augmentation, and non-stereo rendering and indirect augmentation with/without video background. Also, gesture interactions, such as multi-touch interaction and hand gesture-based interaction, were implemented and evaluated. We performed quantitative and qualitative analyses, including performance measurement and questionnaire evaluation. The experimental assessment proves that both FOV and visual registration between virtual and physical artifacts are important, and they can complement each other. Hand gesture-based interaction can be more intuitive and useful. Therefore, by analyzing the advantages and disadvantages of the visual context and gesture interaction in wearable AR, this study suggests more effective and user-centric guidance for task assistance.

show abstract

“…Users can often anticipate and alter their input behavior to avoid auto-correct errors, e.g. by force (Weir et al, 2014), by long pressing a key (Vertanen et al, 2019), or by switching to a precise input mode (Dudley et al, 2018). Similarly, our abbreviated input method needs a way to specify words that should not be expanded or auto-corrected.…”

Section: Discussionmentioning

confidence: 99%

Accelerating Text Communication via Abbreviated Sentence Input

Adhikary¹,

Berger²,

Vertanen³

2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

Self Cite

View full text Add to dashboard Cite

Typing every character in a text message may require more time or effort than strictly necessary. Skipping spaces or other characters may be able to speed input and reduce a user's physical input effort. This can be particularly important for people with motor impairments. In a large crowdsourced study, we found workers frequently abbreviated text by omitting midword vowels. We designed a recognizer optimized for expanding noisy abbreviated input where users often omit spaces and mid-word vowels. We show using neural language models for selecting conversational-style training text and for rescoring the recognizer's n-best sentences improved accuracy. On noisy touchscreen data collected from hundreds of users, we found accurate abbreviated input was possible even if a third of characters was omitted. Finally, in a study where users had to dwell for a second on each key, sentence abbreviated input was competitive with a conventional keyboard with word predictions.

show abstract

Fast and Precise Touch-Based Text Entry for Head-Mounted Augmented Reality with Variable Occlusion

Cited by 50 publications

References 38 publications

Performance Envelopes of Virtual Keyboard Text Input Strategies in Virtual Reality

Performance Envelopes of Virtual Keyboard Text Input Strategies in Virtual Reality

User Interactions for Augmented Reality Smart Glasses: A Comparative Evaluation of Visual Contexts and Interaction Gestures

Accelerating Text Communication via Abbreviated Sentence Input

Contact Info

Product

Resources

About