Music in Extended Realities

Abstract

The intersection between music and Extended Reality (XR) has grown significantly over the past twenty years, amounting to an established area of research today. The use of XR technologies represents a fundamental paradigm shift for various musical contexts as they disrupt traditional notions of musical interaction by enabling performers and audiences to interact musically with virtual objects, agents, and environments. This article both surveys and expands upon the knowledge accumulated in existing research in this area to build a foundation for future works that bring together Music and XR. To this end, we created a freely available dataset of 260 publications in this space and conducted an in-depth analysis covering 199 works in the last decade. We conducted this analysis using a list of conceptual dimensions belonging to technical, artistic, perceptual and methodological domains. This review of the literature is complemented with a set of interviews with domain experts with the goal of establishing a definition for the emergent field of Musical XR, i.e., the field of music in Extended Realities. Based on the results of the conducted review, a research agenda for the field is proposed.

Bayesian Structure Learning and Visualization for Technology Analysis

“…XR technology is being used in various fields such as education, firefighting, culture, music, etc. [2][3][4]6]. In recent years, XR technology has evolved further in cloud computing environments [1].…”

“…Recently, XR technology has been actively researched in various academic fields, and at the same time, demand for it is continuously increasing in industrial fields. In particular, in fields such as manufacturing, cultural industry, education and disaster prevention, XR technology has already been applied to improve the quality of human life [1][2][3][4][5][6]. However, the XR technology is still in beginning stage of research and development, and the possibilities for application of XR technology will continue to increase.…”

“…Extended reality (XR) technology, including virtual reality (VR), augmented reality (AR) and mixed reality (MR), has been rapidly developing in recent years and have a lot of influence on our society in various fields such as education, psychology, firefighting, culture and manufacturing [1][2][3][4][5][6]. As our daily life is increasingly dependent on XR technology, we need to have a good understanding of this technology.…”

Cyberdream: An Interactive Rave Music Visualization in Virtual Reality

“…Post COVID-19 there have also been virtual music festivals such as Tomorrowland 2020, where DJs perform in live-streamed computer-generated environments made in game engines such as the Unreal Development Kit. Whilst these works show some possibilities for visualising music in VR, there has been a call for more software applications demonstrating other new compositional approaches and techniques [Turchet, Hamilton and Çamci 2021]. Cyberdream responds to this call by exploring how VJ practices can be translated into the interactive spatial modalities of VR.…”

Synaesthetic Audio-Visual Sound Toys in Virtual Reality

“…Audio-visual performance is a well-established field, which encompasses a range of creative practices. Within this area, virtual reality (VR) music visualization is a growing field [7], which includes many interesting opportunities for new forms of audio-visual composition and experience to emerge. In Rajmil Fischman's discussion of 'music in the holodeck' [4], he calls for accessible, interactive experiences that allow participants to create electronic sound in immersive environments.…”