The Cost of Production in Elicitation Studies and the Legacy Bias-Consensus Trade off

Abstract

Gesture elicitation studies are a popular means of gaining valuable insights into how users interact with novel input devices. One of the problems elicitation faces is that of legacy bias, when elicited interactions are biased by prior technologies use. In response, methodologies have been introduced to reduce legacy bias. This is the first study that formally examines the production method of reducing legacy bias (i.e., repeated proposals for a single referent). This is done through a between-subject study that had 27 participants per group (control and production) with 17 referents placed in a virtual environment using a head-mounted display. This study found that over a range of referents, legacy bias was not significantly reduced over production trials. Instead, production reduced participant consensus on proposals. However, in the set of referents that elicited the most legacy biased proposals, production was an effective means of reducing legacy bias, with an overall reduction of 11.93% for the chance of eliciting a legacy bias proposal.

Iterative Design of Gestures During Elicitation: Understanding the Role of Increased Production

“…However, they categorized gestures as being infuenced by legacy bias only if participants verbally mentioned they were infuenced by prior technology during the study. In comparison, Williams et al [55] categorized a proposed gesture as a legacy or non-legacy gesture through a consensus of independent votes of three expert raters if the gesture class could be identifed as being used with a known device in common usage. In their study, they found that increased production reduced consensus on proposals and was effective in reducing legacy bias by about 12% for those referents that were most likely to be infuenced by bias in the frst place.…”

“…[33]. Existing research has produced contradictory outcomes on the efect of increased production on legacy gestures [20,33,55]. We ofer a slightly diferent interpretation: that those minor variations are not irrelevant, and that these latter proposals are refnements of prior gestures which preserve the gestural features that matter most to users.…”

“…As prior work has produced such contradicting outcomes on the efect of increased production on legacy gestures [20,33,55], we take a closer look at the efects of unlimited production on user preference and on legacy bias in this study. We start with user preference.…”

“…Some studies showed little to change in the legacy gestures participants produced [20], and others resulted in participants producing more novel gestures [33]. Yet other studies found that the efect of production reduced legacy bias in those referents that were most prone to legacy biased proposals [55]. From these results, more research must be conducted to understand the impact of increased production on legacy bias.…”

User-Defined Interactions for Visual Data Exploration With the Combination of Smartwatch and Large Display

“…Many of the atomic gestures, such as the dragging and splay gesture for T-19 and T-1, are taken from the screen-based interactions. As demonstrated by a fair number of existing studies, legacy bias is a contributing factor that users generate personally satisfying designs [58], [59], [60]. Due to the conscious use of previous knowledge [61], the designs that successfully leverage legacy bias can have a positive impact on their performance.…”