Emotion-prints: interaction-driven emotion visualization on multi-touch interfaces

Abstract

Emotions are one of the unique aspects of human nature, and sadly at the same time one of the elements that our technological world is failing to capture and consider due to their subtlety and inherent complexity. But with the current dawn of new technologies that enable the interpretation of emotional states based on techniques involving facial expressions, speech and intonation, electrodermal response (EDS) and brain-computer interfaces (BCIs), we are finally able to access real-time user emotions in various system interfaces. In this paper we introduce emotion-prints, an approach for visualizing user emotional valence and arousal in the context of multi-touch systems. Our goal is to o↵er a standardized technique for representing user a↵ective states in the moment when and at the location where the interaction occurs in order to increase a↵ective self-awareness, support awareness in collaborative and competitive scenarios, and o↵er a framework for aiding the evaluation of touch applications through emotion visualization. We show that emotion-prints are not only independent of the shape of the graphical objects on the touch display, but also that they can be applied regardless of the acquisition technique used for detecting and interpreting user emotions. Moreover, our representation can encode any a↵ective information that can be decomposed or reduced to Russell's two-dimensional space of valence and arousal. Our approach is enforced by a BCI-based user study and a follow-up discussion of advantages and limitations.

EMOTE: An interactive online tool for designing real-time emotional AR visualizations

“…Creating emotion visualizations is a well-investigated research topic in HCI. Researchers have worked with various mediums for presenting visualizations; integrated into apps [5,13,21], wearables [1,20], fashion pieces [30,32], using both 2D [5,13,21] and 3D [3,19,23,28,31,33] visualizations.…”

“…In most cases, visualizations are static, representing one emotional state at a time, and have contextual associations (i.e. UI elements [5], location and time [13], digital behavior such as tasks [21]). The focus on real-time emotion visualization is limited.…”

“…However, how affective information should be visualized in AR is not as clear. There is a variety of works on emotion representation in the HCI literature (some examples are [3,31,37]), designed to be integrated with existing applications and 2D interfaces [5], or conceptualized as wearables [1], but they are not targeted for AR. This paper focuses on the design space of expressive emotion visualizations that can appear around one's body as they experience the referenced emotional states.…”

AffectiveTDA: Using Topological Data Analysis to Improve Analysis and Explainability in Affective Computing

“…The widgets employed emotion scents, hue-varied colormaps representing either valance or arousal, e.g., red and green represent negative and positive valance, respectively. Emotion-prints was an early system to provided real-time feedback of valance and arousal to users using touch-displays [10]. More recently, Kovacevik et al [47] employed ideas from SAM and emotion scents to create a glyph for simultaneous representation of valence and arousal.…”

Visualizing Excitement of Individuals and Groups

“…In terms of real-time visualization of emotions, Cernea et al [7] introduced Emotion-prints, a visualization system for touch-enabled interfaces, where the current level of a user's valence and arousal (estimated through EEG measurements) is represented in the shape of a animated halo around the virtual objects that the user touches. Saari et al [29] introduced a mobile emotion visualization system devised for improving group performance and awareness.…”