A Hypothesis of Brain-to-Brain Coupling in Interactive New Media Art and Games Using Brain-Computer Interfaces

Abstract

Abstract.Interactive new media art and games belong to distinctive fields, but nevertheless share common grounds, tools, methodologies, challenges, and goals, such as the use of applications and devices for engaging multiple participants and players, and more recently electroencephalography (EEG)-based brain-computer interfaces (BCIs). At the same time, an increasing number of new neuroscientific studies explore the phenomenon of brain-tobrain coupling, the dynamics and processes of the interaction and synchronisation between multiple subjects and their brain activity. In this context, we discuss interactive works of new media art, computer and serious games that involve the interaction of the brain-activity, and hypothetically brain-to-brain coupling, between multiple performer/s, spectator/s, or participants/players. We also present Enheduanna -A Manifesto of Falling (2015), a new live brain-computer cinema performance, with the use of an experimental passive multi-brain BCI system under development. The aim is to explore brain-to-brain coupling between performer/s and spectator/s as means of controlling the audio-visual creative outputs.

“Enheduanna—A Manifesto of Falling” Live Brain-Computer Cinema Performance: Performer and Audience Participation, Cognition and Emotional Engagement Using Multi-Brain BCI Interaction

“…However, earlier, as well as recent examples, involve the simultaneous interaction of more than one participants or performers with the use of EEG-based multi-brain BCIs (Nijholt, 2015 ; Zioga et al, 2016 ). This trend coincides with the increasing interest in the fields of neuroscience and experimental psychology “in studying the mechanisms, dynamics, and processes of the interaction between multiple subjects and their brain-activity” (Zioga et al, 2015 ) and even more in a real-life context, away from the lab.…”

“…The participants' raw EEG data are acquired simultaneously with the MyndPlay Brain-BandXL EEG Headset, which has two dry sensors located in the prefrontal cortex, with one passive (Fpz), one active (Fp1) and a grounding electrode on the ear lobe. The data are transmitted wirelessly to the main computer via Bluetooth (Zioga et al, 2015 , 2016 ). The real-time digital recording and processing is performed with the OpenViBE software (Renard et al, 2010 ) using algorithms that follow the frequency analysis method and a custom-based feature extraction.…”

“…In this frame, we discuss and evaluate Enheduanna—A Manifesto of Falling , a live brain-computer cinema performance—that is an interactive performative work that combines live, mediatised representations, more specifically live cinema, and the use of BCIs (Zioga et al, 2015 , 2016 ). The performance was realised as a complete combination of scientific and creative practice-based methodologies and solutions, addressing the challenges of the design and implementation of multi-brain BCIs in mixed-media performances with audience participation.…”

Multi-Brain BCI: Characteristics and Social Interactions

“…In the first situation we are interested in understanding the group process and ways to improve it; this may concern decision making, but it can also concern less task-oriented issues such as social cohesion and empathy awareness; in that way it becomes part of social interaction research. In the second situation we are interested in having one or more persons' brain activity getting involved in creating or adapting a piece of life media art, maybe also in interaction with an artist/performer who is coordinating the joint life performance [14]. In the third situation we are interested in the statistics that can be obtained and from analysis we learn about preferences that help to improve the product.…”

Enheduanna–A Manifesto of Falling: first demonstration of a live brain-computer cinema performance with multi-brain BCI interaction for one performer and two audience members

“…In this context, we present in Section 2 the neuroscientific, computational, creative, performative and experimental challenges of the design and implementation of multi-brain BCIs in mixedmedia performances. Accordingly, we discuss in Section 3 Enheduanna -A Manifesto of Falling, the first demonstration of a live brain-computer cinema performance, as a complete combination of creative and research solutions, which is based on our previous work (Zioga et al 2014(Zioga et al , 2015 with the use of BCIs in performances that involve audience participation and interaction with a performer (Nijholt and Poel 2016, 81). This new work enables for the first time, to our present knowledge, the simultaneous real-time interaction with the use of EEG of more than two participants, including both a performer as well as members of the audience in the context of a mixed-media performance.…”