Touchless interfaces allow users to view, control and manipulate digital content without physically touching an interface. They are being explored in a wide range of application scenarios from medical surgery to car dashboard controllers. One aspect of touchless interaction that has not been explored to date is the Sense of Agency (SoA). The SoA refers to the subjective experience of voluntary control over actions in the external world. In this paper, we investigated the SoA in touchless systems using the intentional binding paradigm. We first compare touchless systems with physical interactions and then augmented different types of haptic feedback to explore how different outcome modalities influence users' SoA.From our experiments, we demonstrated that an intentional binding effect is observed in both physical and touchless interactions with no statistical difference. Additionally, we found that haptic and auditory feedback help to increase SoA compared with visual feedback in touchless interfaces. We discuss these findings and identify design opportunities that take agency into consideration.

No abstract

Multisensory integration research has allowed us to better understand how humans integrate sensory information to produce a unitary experience of the external world. However, this field is often challenged by the limited ability to deliver and control sensory stimuli, especially when going beyond audio–visual events and outside laboratory settings. In this review, we examine the scope and challenges of new technology in the study of multisensory integration in a world that is increasingly characterized as a fusion of physical and digital/virtual events. We discuss multisensory integration research through the lens of novel multisensory technologies and, thus, bring research in human–computer interaction, experimental psychology, and neuroscience closer together. Today, for instance, displays have become volumetric so that visual content is no longer limited to 2D screens, new haptic devices enable tactile stimulation without physical contact, olfactory interfaces provide users with smells precisely synchronized with events in virtual environments, and novel gustatory interfaces enable taste perception through levitating stimuli. These technological advances offer new ways to control and deliver sensory stimulation for multisensory integration research beyond traditional laboratory settings and open up new experimentations in naturally occurring events in everyday life experiences. Our review then summarizes these multisensory technologies and discusses initial insights to introduce a bridge between the disciplines in order to advance the study of multisensory integration.

Human–computer integration is an emerging area in which the boundary between humans and technology is blurred as users and computers work collaboratively and share agency to execute tasks. The sense of agency (SoA) is an experience that arises by a combination of a voluntary motor action and sensory evidence whether the corresponding body movements have somehow influenced the course of external events. The SoA is not only a key part of our experiences in daily life but also in our interaction with technology as it gives us the feeling of “I did that” as opposed to “the system did that,” thus supporting a feeling of being in control. This feeling becomes critical with human–computer integration, wherein emerging technology directly influences people’s body, their actions, and the resulting outcomes. In this review, we analyse and classify current integration technologies based on what we currently know about agency in the literature, and propose a distinction between body augmentation, action augmentation, and outcome augmentation. For each category, we describe agency considerations and markers of differentiation that illustrate a relationship between assistance level (low, high), agency delegation (human, technology), and integration type (fusion, symbiosis). We conclude with a reflection on the opportunities and challenges of integrating humans with computers, and finalise with an expanded definition of human–computer integration including agency aspects which we consider to be particularly relevant. The aim this review is to provide researchers and practitioners with guidelines to situate their work within the integration research agenda and consider the implications of any technologies on SoA, and thus overall user experience when designing future technology.

The Sense of Agency (SoA) refers to our capability to control our own actions and influence the world around us. Recent research in HCI has been exploring SoA to provide users an instinctive sense of "I did that" as opposed to "the system did that". However, current agency measurements are limited. The Intentional Binding (IB) paradigm provides an implicit measure of the SoA. However, it is constrained by requiring high visual attention to a "Libet clock" onscreen. In this paper, we extend the timing stimulus through auditory and tactile cues. Our results demonstrate that audio timing through voice commands and haptic timing through tactile cues on the hand are alternative techniques to measure the SoA using the IB paradigm. They both address limitations of the traditional method (e.g., lack of engagement and visual demand). We discuss how our results can be applied to measure SoA in tasks involving different interactive scenarios common in HCI.

It has long been known that our sense of smell is a powerful one that affects emotions and behaviors. Recently, interest in the sense of smell has been growing exponentially in HCI. However, the potential of smell to inspire design is still underexplored. In this paper, we first investigated crossmodal correspondences between scents and selected features relevant for design (clustered in sensory, bodily, and qualitative features). Then, we created a set of cards (EssCards) to visually summarize the key findings to inspire designers. We carried out two preliminary design exploration sessions using the EssCards. Based on our findings, we discuss how to inspire and challenge design opportunities around the sense of smell and reflect upon applications for smell as inspirational material for designing future interactions and experiences.

The Sense of Agency (SoA) is crucial in interaction with technology, it refers to the feeling of 'I did that' as opposed to 'the system did that' supporting a feeling of being in control. Research in human-computer interaction has recently studied agency in visual, auditory and haptic interfaces, however the role of smell on agency remains unknown. Our sense of smell is quite powerful to elicit emotions, memories and awareness of the environment, which has been exploited to enhance user experiences (e.g., in VR and driving scenarios). In light of increased interest in designing multimodal interfaces including smell and its close link with emotions, we investigated, for the first time, the effect of smellinduced emotions on the SoA. We conducted a study using the Intentional Binding (IB) paradigm used to measure SoA while participants were exposed to three scents with different valence (pleasant, unpleasant, neutral). Our results show that participants' SoA increased with a pleasant scent compared to neutral and unpleasant scents. We discuss how our results can inform the design of multimodal and future olfactory interfaces.

TESTING & TUNING OUR CORRECTION TECHNIQUESWe conducted paired comparisons with Bonferroni corrections to compare the Drift effect with both correction techniques (Derivative and Angular) in each configuration of α and ε (49 configurations in total). Table S1 shows the mean and p values of this comparison. Results suggest a poor performance of the Derivative correction when compared with the uncorrected Drift. Averages with SD of all the configuration are shown in Figure S1. α ε Uncorrected Derivative Angular Uncorrected Vs Derivative Uncorrected Vs Angular Derivative Vs Angular 0.125 0.35 0.566 m 0.665 m 0.374 m p=0.037 p<0.001 p<0.