Boiling Mind: Amplifying the Audience-Performer Connection through Sonification and Visualization of Heart and Electrodermal Activities

Sugawa, Moe; Furukawa, Taichi; Chernyshov, George; Hynds, Danny; Han, Jiawen; Padovani, Marcelo; Zheng, Dingding; Marky, Karola; Kunze, Kai; Minamizawa, Kouta

doi:10.1145/3430524.3440653

Cited by 22 publications

(24 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[77], [89], [90], [91], [118], [133], [136] Co-located [2,63,93] , [17], [32], [40], [46], [54], [87], [88], [100], [110], [116], [121], [122], [123], [142] [37], [55], [98], [112], [119], [120], [83], [130] Synchronous Asynchronous…”

Section: Physical Contextmentioning

confidence: 99%

“…Moreover, biofeedback access was granted to both primary and secondary users in 11 studies [37,63,64,80,93,98,109,112,118,120]. Finally, the four studies designed for both symmetrical and asymmetrical interaction styles involved two or more symmetrically-engaged users and bystanders who could observe the interaction [32,33,122] and provided behavioral feedback [123].…”

Section: Articulating the Physiological-social Interaction Spacementioning

confidence: 99%

See 1 more Smart Citation

Shared User Interfaces of Physiological Data: Systematic Review of Social Biofeedback Systems and Contexts in HCI

Clara

Wang

Cho

2022

CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

As an emerging interaction paradigm, physiological computing is increasingly being used to both measure and feed back information about our internal psychophysiological states. While most applications of physiological computing are designed for individual use, recent research has explored how biofeedback can be socially shared between multiple users to augment human-human communication. Reflecting on the empirical progress in this area of study, this paper presents a systematic review of 64 studies to characterize the interaction contexts and effects of social biofeedback systems. Our findings highlight the importance of physio-temporal and social contextual factors surrounding physiological data sharing as well as how it can promote social-emotional competences on three different levels: intrapersonal, interpersonal, and task-focused. We also present the Social Biofeedback Interactions framework to articulate the current physiological-social interaction space. We use this to frame our discussion of the implications and ethical considerations for future research and design of social biofeedback interfaces.CCS CONCEPTS • Human-centered computing → Human computer interaction (HCI): Interaction techniques; Interactive systems and tools; Interaction paradigms; Interaction devices; HCI theory, concepts and models.

show abstract

Section: Physical Contextmentioning

confidence: 99%

Section: Articulating the Physiological-social Interaction Spacementioning

confidence: 99%

Shared User Interfaces of Physiological Data: Systematic Review of Social Biofeedback Systems and Contexts in HCI

Clara

Wang

Cho

2022

CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

show abstract

“…The Boiling Mind [22] project showed a novel way of choreography stage production which enhances performer-audience connection through sonification and visualization of physiological sensing data. Boiling Mind also inspired this research a lot by blurring the boundaries between performers and audience and bringing the audience to the stage.…”

Section: Related Work Of Neurological Aestheticsmentioning

confidence: 99%

Frisson Waves: Sharing Frisson to Create Collective Empathetic Experiences for Music Performances

Chernyshov

Zheng

et al. 2021

SIGGRAPH Asia 2021 Emerging Technologies

Self Cite

View full text Add to dashboard Cite

Frisson is a feeling and a mental experience of body reactions such as shivers, tingling skin, goosebumps. This research proposes Frisson Waves, a real-time system to detect, trigger and share frisson in a wave-like pattern during music performances. The system consists of a physiological sensing wristband for detecting frisson and a thermo-haptic neckband for inducing frisson. This project aims to improve the connectedness of audience members and performers during music performances by sharing frisson. In this thesis, we described thoroughly the literature and works related to exteroception and interoception interaction, emotion contagion, concept design process, and system development process. Results of studies proved a significant effect of triggering frisson, detecting frisson and capability of sharing frisson and primitive enhancement of connectedness in a live performance context.

show abstract

“…For example, “happier” music was associated with a “happy” color (yellow). Musical features also played a role, such as slower music associated with the color blue, because it was mediated by the low arousal emotion of sadness (see also Whiteford et al, 2018 ; Sugawa et al, 2021 ). Schubert et al (2019) asked participants to describe pieces of music with a number of icon-based rating scales, including representations of temperature, color, feelings, and shapes.…”

Section: Introductionmentioning

confidence: 99%

Illuminating Music: Impact of Color Hue for Background Lighting on Emotional Arousal in Piano Performance Videos

et al. 2022

View full text Add to dashboard Cite

This study sought to determine if hues overlayed on a video recording of a piano performance would systematically influence perception of its emotional arousal level. The hues were artificially added to a series of four short video excerpts of different performances using video editing software. Over two experiments 106 participants were sorted into 4 conditions, with each viewing different combinations of musical excerpts (two excerpts with nominally high arousal and two excerpts with nominally low arousal) and hue (red or blue) combinations. Participants rated the emotional arousal depicted by each excerpt. Results indicated that the overall arousal ratings were consistent with the nominal arousal of the selected excerpts. However, hues added to video produced no significant effect on arousal ratings, contrary to predictions. This could be due to the domination of the combined effects of other channels of information (e.g., the music and player movement) over the emotional effects of the hypothesized influence of hue on perceived performance (red expected to enhance and blue to reduce arousal of the performance). To our knowledge this is the first study to investigate the impact of these hues upon perceived arousal of music performance, and has implications for musical performers and stage lighting. Further research that investigates reactions during live performance and manipulation of a wider range of lighting hues, saturation and brightness levels, and editing techniques, is recommended to further scrutinize the veracity of the findings.

show abstract

Boiling Mind: Amplifying the Audience-Performer Connection through Sonification and Visualization of Heart and Electrodermal Activities

Cited by 22 publications

References 33 publications

Shared User Interfaces of Physiological Data: Systematic Review of Social Biofeedback Systems and Contexts in HCI

Shared User Interfaces of Physiological Data: Systematic Review of Social Biofeedback Systems and Contexts in HCI

Frisson Waves: Sharing Frisson to Create Collective Empathetic Experiences for Music Performances

Illuminating Music: Impact of Color Hue for Background Lighting on Emotional Arousal in Piano Performance Videos

Contact Info

Product

Resources

About