Can Biosignals be Expressive?

Abstract

We are exploring the concept of expressive biosignals: leveraging wearable technologies to introduce sensed physiological data as cues for social perception. Biosignals can help us achieve a deeper understanding of each other by revealing or clarifying the psychological processes that underlie our subjective experience. We conducted an exploratory study investigating expressive biosignals, comparing the influence of a variety of brain activity visualizations on impression formation. Results revealed that while participants readily infer emotional and cognitive states from visualized brain activity, the ambiguity of the data can lead to diverse perceptions and interpretations. Participants also expressed concerns that the observation of another individual's data during interaction might be invasive or distracting. We present a set of design considerations addressing issues of interpretability, integration, and privacy of biosignals in interpersonal contexts.

The Effect of Expressive Biosignals on Empathy and Closeness for a Stigmatized Group Member

“…Prior literature suggests the potential for expressive biosignals to affect empathy for others. In particular, several qualitative studies that deploy expressive biosignal systems (both in the wild and in laboratory settings) have demonstrated that people tend to infer and be more aware of a close contact's emotional state through displays of heart rate and skin conductance [35,38,49,51].…”

“…Research has also shown that expressive biosignals can affect closeness with others. For instance, systems such as rings or smartwatches where you can feel or see a partner's heart rate can improve feelings of being connected with remote friends and family [49,51,56,63,69]. Janssen and colleagues demonstrate that biosignals can also increase feelings of closeness with strangers.…”

“…At the same time, different ways of conveying biosignals information could have differing effects on closeness. In a mixed-methods experiment involving six brain activity visualizations, Liu and colleagues show that different visualizations can lead to different feelings of connectedness with a stranger [49]. Moreover, in a study where participants sent texts of their heart rate to others, Liu and colleagues found that participants desired more vivid and expressive visualizations for their heart rate than simple text denoting their beats per minute [50].…”

“…Second, a graph would help control the information conveyed to participants. Prior work shows that design characteristics of different biosignals visualizations can have a significant effect on interpersonal perceptions [49]. For instance, perceptions could be confounded by prior associations people have with representations like an emoji or audio or vibrations of a heartbeat, which are all culturally embedded.…”

“…Potentially, the visualization we chose for heart rate, a graph, did not provide a vivid representation of Jared's feelings. Indeed, the form of a screen-based graph of raw data may be too "clinical," as compared to more physical or abstract forms such as a colored light bulb [49] or worn watch with haptic feedback [51]. Future work should explore which visualization schemes for heart rate best provide cues that could increase the overall salience of the other person.…”