Agency in Mid-air Interfaces

Abstract

Touchless interfaces allow users to view, control and manipulate digital content without physically touching an interface. They are being explored in a wide range of application scenarios from medical surgery to car dashboard controllers. One aspect of touchless interaction that has not been explored to date is the Sense of Agency (SoA). The SoA refers to the subjective experience of voluntary control over actions in the external world. In this paper, we investigated the SoA in touchless systems using the intentional binding paradigm. We first compare touchless systems with physical interactions and then augmented different types of haptic feedback to explore how different outcome modalities influence users' SoA.From our experiments, we demonstrated that an intentional binding effect is observed in both physical and touchless interactions with no statistical difference. Additionally, we found that haptic and auditory feedback help to increase SoA compared with visual feedback in touchless interfaces. We discuss these findings and identify design opportunities that take agency into consideration.

May the Force Be with You

“…It is necessary to provide effective feedback to mid-air gestures to help users overcome uncertainty [12]. It reduces mental workload [39,32] and provides a sense of control of the outcomes of one's actions [8]. It decreases eyes-off-the-road time [41] and improves user satisfaction [33].…”

“…Ultrasound haptics uses acoustic radiation pressure to create areas of friction on the skin and thus tactile sensations. It offers the new opportunity of "adding touch to a touchless interaction" [13] and therefore increases the sense of control over the user's intended mid-air gesture actions [8]. Additionally, it conveys tactile feedback which can be felt by the unadorned hand in mid-air mitigating the problem of on-body-displays.…”

“…Auditory feedback in driving environments has been shown to reduce looking away time [10] and if presented to mid-air gestures it reduces eyes-off-the-road time without negatively impacting the driving performance [41,43] nor mental demand [41,32]. A multimodal combination of ultrasound and audio feedback can surpass the shortcomings of ultrasound haptics for mid-air feedback [8].…”

“…For instance, the driver's unfamiliarity with a gesture system has a negative impact on lane keeping ability and mental workload [19,38]. The decoupling of the user's hand from the interface [7] and the lack of sense of control over the touch less interface [8] are additional aspects which lessen user satisfaction, safety and usability of gestures [33].…”

“…Previous works have shown that there is growing interest in ultrasound feedback for mid-air gesture interaction in cars [40,8,17]. Rümelin et al [40] investigated ultrasound feedback for mid-air pointing.…”