Designing with Gaze

Abstract

Recent developments in gaze tracking present new opportunities for social computing. This paper presents a study of Tama, a gaze actuated smart speaker. Tama was designed taking advantage of research on gaze in conversation. Rather than being activated with a wake word (such as "Ok Google") Tama detects the gaze of a user, moving an articulated 'head' to achieve mutual gaze. We tested Tama's use in a multi-party conversation task, with users successfully activating and receiving a response to over 371 queries (over 10 trials). When Tama worked well, there was no significant difference in length of interaction. However, interactions with Tama had a higher rate of repeated queries, causing longer interactions overall. Video analysis lets us explain the problems users had interacting with gaze. In the discussion, we describe implications for designing new gaze systems, using gaze both as input and output. We also discuss how the relationship to anthropomorphic design and taking advantage of learned skills of interaction. Finally, two paths for future work are proposed, one in the field of speech agents, and the second in using human gaze as an interaction modality more widely.

Machine Body Language: Expressing a Smart Speaker’s Activity with Intelligible Physical Motion

“…all LEDs turn yellow) indicates that the prototype is ready to attend to the user's requests. Yellow is associated with the 'ready' state, as has been argued and used by McMillan et al [52]. Additionally, the bright yellow blends well with other light sources in a typical home environment.…”

“…Such presence informs the user about their IPA's location and if it is attentive to their requests, which helps the user determine whether the IPA is potentially violating their privacy or not, or if they are attentive, similar to how we perceive another person's presence. Similarly, McMillan et al [52] designed Tama, a gaze-aware smart speaker, which is invoked through mutual gaze. In combination with head rotation, Tama is capable of establishing mutual gaze and orient itself towards the user with which it is interacting.…”

“…parts of the LEDs turn blue and start moving in a circular motion) indicates that the prototype is busy searching for a response, hence not ready to attend to the user's request. Similar to yellow, we were inspired by McMillan et al 's use of blue in a "loss of gaze" context [52], which we translated here as a loss of attention, i.e. the busy state.…”

Measuring Voice UX Quantitatively

“…Then, they conducted two inductive thematic analyses [5,16] to create categorical frameworks for each of the IV and DV data. First, they independently developed basic frameworks, then [39]; [20] combined them through discussion, and then applied them to all 29 papers. Two rounds of coding were conducted for inter-rater reliability to be achieved, with Kappa values of .80+ per category.…”

“…It applies to "bodyless" and ever-present voice systems, where there is not necessarily a visual or physical interface to begin or conduct interactions. The table also [39]; Exercise Behavior [37]; Gaze [16]; Language production (e.g., lexical complexity, adaptation) [51]; Interaction time [27]; Driving performance [38]; Disclosure (user responses) [53]; Accidents [41] Subj…”