Spatial Interfaces and Interactive 3D Environments for Immersive Musical Performances

Abstract

The power of interactive 3D graphics, immersive displays, and spatial interfaces is still under-explored in domains where the main target is to enhance creativity and emotional experiences. This article presents a set of work the attempts to extent the frontiers of music creation as well as the experience of audiences attending to digital performances. The goal is to connect sounds to interactive 3D graphics that musicians can interact with and the audience can observe.

Mobile virtual reality for musical genre learning in primary education

“…In [23], an abstract environment is presented where performers can collaboratively create music in VR. Berthaut and co-workers [24] have also proposed several immersive interfaces for musical performances, arguing how the power of interactive 3D graphics, immersive displays, and spatial interfaces is still under-explored in domains where the main target is to enhance creativity and emotional experiences.…”

ICT in the Teacher Training System for Academic Vocals: Structured Mentoring

“…Thus, in modern studies [13], devoted to the analysis of the concepts of immersive media reality in the context of the methodological support of teacher training in spatial interfaces and interactive 3D environment for immersive musical performances, immersive reproduction and spatial interfaces are considered as topics poorly studied in areas focused on developing creativity and enhancing emotional experience. A number of works [17,13] are devoted to modifying ICT technologies, expanding the boundaries of creating musical works, as well as the experience of performing in digital format where the goal is to integrate sound and interactive 3D graphics that musicians, vocalists can work with and viewers can watch.…”

Musical Metaverse: vision, opportunities, and challenges

“…The use of XR technologies in musical activities represents a paradigm shift as they disrupt traditional notions of musical interaction by enabling performers and audiences to interact musically with virtual/augmented objects, agents, and environments. A large variety of musical activities have been investigated using the XR medium: systems have been devised for supporting composition (e.g., [22][23][24]), education (e.g., [25,26]), performance (e.g., [27][28][29]), entertainment (e.g., [30][31][32]), and sound engineering (e.g., [33][34][35][36]. Moreover, various software and hardware tools have been devised to support the development of Musical XR systems (e.g., [37][38][39]39]), along with design and analysis frameworks [41], while different studies have investigated the perception in Musical XR settings (e.g., [42,43]).…”