Modeling Human Performance of Stroke-Based Text Entry

Gesture-typing is an efficient, easy-to-learn, and errortolerant technique for entering text on software keyboards. Our goal is to "recycle" users' otherwise-unused gesture variation to create rich output under the users' control, without sacrificing accuracy. Experiment 1 reveals a high level of existing gesture variation, even for accurate text, and shows that users can consciously vary their gestures under different conditions. We designed an Expressive Keyboard for a smart phone which maps input gesture features identified in Experiment 1 to a continuous output parameter space, i.e. RGB color. Experiment 2 shows that users can consciously modify their gestures, while retaining accuracy, to generate specific colors as they gesture-type. Users are more successful when they focus on output characteristics (such as red) rather than input characteristics (such as curviness). We designed an app with a dynamic font engine that continuously interpolates between several typefaces, as well as controlling weight and random variation. Experiment 3 shows that, in the context of a more ecologically-valid conversation task, users enjoy generating multiple forms of rich output. We conclude with suggestions for how the Expressive Keyboard approach can enhance a wide variety of gesture recognition applications.

show abstract

“…typef ace interpolated = n j=1 typeface j weight j n j=1 weight j (5) where each typeface is a vector of control points for each letter.…”

Section: Methodsmentioning

confidence: 99%

“…Speed is a common measure of gesture variation, e.g., for typing activity [19,8] and modeling the production time of a gesture [5,7]. For gestures, speed is calculated by dividing the total length of traced distance (in pixels) by the total time (in milliseconds).…”

Section: Speedmentioning

confidence: 99%

Expressive Keyboards

Alvina

Malloch

Mackay

2016

Proceedings of the 29th Annual Symposium on User Interface Software and Technology

View full text Add to dashboard Cite

show abstract

“…Table 2 provides a brief description of each feature. For the practical purposes of our evaluation, we considered a set of 18 features reported by prior work as key to evaluate user performance with gesture input (features 1-5), inform gesture set design (6)(7)(8), and recognize stroke gestures (6,(9)(10)(11)(12)(13)(14)(15)(16)(17)(18) Note that these feature categories are not mutually exclusive. For example, production_time is also a good estimator of users' perceived difficulty to articulate stroke gestures [81,103].…”

Section: Gesture Featuresmentioning

confidence: 99%

“…For example, compared to traditional approaches, a gesture-based password enables users to authenticate faster on mobile devices and, moreover, users are more willing to retry entering the password in case of erroneous input [112]. Ultimately, research on stroke gesture input has made possible new text entry techniques for mobile devices [16,82], such as shape-writing or gesture-typing [45,46], widely available on today's smartphones and influencing research and development of future text entry methods for wearable devices [21,32,113].…”

Section: Introductionmentioning

confidence: 99%

Omnis Prædictio: Estimating the full spectrum of human performance with stroke gestures

Leiva¹,

Vatavu²,

Martín-Albo³

et al. 2020

International Journal of Human-Computer Studies

View full text Add to dashboard Cite

Designing effective, usable, and widely adoptable stroke gesture commands for graphical user interfaces is a challenging task that traditionally involves multiple iterative rounds of prototyping, implementation, and follow-up user studies and controlled experiments for evaluation, verification, and validation. An alternative approach is to employ theoretical models of human performance, which can deliver practitioners with insightful information right from the earliest stages of user interface design. However, very few aspects of the large spectrum of human performance with stroke gesture input have been investigated and modeled so far, leaving researchers and practitioners of gesture-based user interface design with a very narrow range of predictable measures of human performance, mostly focused on estimating production time, of which extremely few cases delivered accompanying software tools to assist modeling. We address this problem by introducing "Omnis Praedictio" (Omnis for short), a generic technique and companion web tool that provides accurate user-independent estimations of any numerical stroke gesture feature, including custom features specified in code. Our experimental results on three public datasets show that our model estimations correlate on average r s > .9 with groundtruth data. Omnis also enables researchers and practitioners to understand human performance with stroke gestures on many levels and, consequently, raises the bar for human performance models and estimation techniques for stroke gesture input.

show abstract

Synthetic Word Gesture Generation for Stroke-Based Virtual Keyboards

Burgbacher

Hinrichs

2017

IEEE Trans. Human-Mach. Syst.

View full text Add to dashboard Cite

Modeling Human Performance of Stroke-Based Text Entry

Cited by 4 publications

References 18 publications

Expressive Keyboards

Expressive Keyboards

Omnis Prædictio: Estimating the full spectrum of human performance with stroke gestures

Synthetic Word Gesture Generation for Stroke-Based Virtual Keyboards

Contact Info

Product

Resources

About