Affective Visualization in Virtual Reality: An Integrative Review

Abstract

A cluster of research in Affective Computing suggests that it is possible to infer some characteristics of users’ affective states by analyzing their electrophysiological activity in real-time. However, it is not clear how to use the information extracted from electrophysiological signals to create visual representations of the affective states of Virtual Reality (VR) users. Visualization of users’ affective states in VR can lead to biofeedback therapies for mental health care. Understanding how to visualize affective states in VR requires an interdisciplinary approach that integrates psychology, electrophysiology, and audio-visual design. Therefore, this review aims to integrate previous studies from these fields to understand how to develop virtual environments that can automatically create visual representations of users’ affective states. The manuscript addresses this challenge in four sections: First, theories related to emotion and affect are summarized. Second, evidence suggesting that visual and sound cues tend to be associated with affective states are discussed. Third, some of the available methods for assessing affect are described. The fourth and final section contains five practical considerations for the development of virtual reality environments for affect visualization.

Mood Worlds: A Virtual Environment for Autonomous Emotional Expression

“…Other ways of measuring the induction of emotions encompass behavioural measures such as body movement, voice pattern, facial expression, or physiological signals, e.g. heart rate, brain activity or electrodermal activity (for an overview, see [59]). Although these measures could provide more objective data than self-reports, most are time-consuming, expensive, and often too complex in their usability for people to use without assistance of experts.…”

“…Other designers fostered autonomy by actively engaging participants during the design process of a VE, but the design could not be further adjusted after entering the VE [28]. However, research suggests that the process of autonomously designing VEs is therapeutic in itself [84] and that emotional content should be adjusted in real-time [59].…”

“…Further research has looked at the impact of specific elements in virtual scenarios [59,65]. Regarding visual cues, higher saturated and bright colours are more arousing and pleasant [44,54,55], fast-moving objects induce a higher arousal [24,60], and natural settings are perceived as more relaxing [50,83].…”

“…However, in most cases not one specific element but a synergy of colour, sound, lighting, weather, setting, characters, animals and objects induce a certain affective state [23]. Thus, Pinilla et al [59] have developed broader guidelines for designing visual content of VR applications used to elicit positive change in their users. Moreover, there might be intra-and interpersonal differences of how the same audio-visual stimuli are perceived regarding affective states, also related to personal experiences [59].…”

“…Furthermore, caring for one's well-being and mental health at home and designing technologies supporting that endeavour has gained in importance [70,81,89]. Thus, emotionally stimulating content should be adjusted in real-time if necessary [59]. Recent work suggests that becoming active in designing emotionally stimulating content is in itself therapeutic and can foster happiness [84].…”