Web-Based Networked Music Performances via WebRTC: A Low-Latency PCM Audio Solution

Sacchetto, Matteo; Gastaldi, Paolo; Chafe, Chris; Rottondi, Cristina; Servetti, Antonio

doi:10.17743/jaes.2022.0021

Cited by 9 publications

(8 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a result, the current implementation is more suited to realizing novel and tailor-made music pieces and performances than transforming existing material. Even under good conditions, latencies in standard WebRTC audio streams can be perceived as too high (>60 ms) for latency-critical NMPs [89]; this is also true for volumetric video streaming [90].…”

Section: Discussionmentioning

confidence: 99%

“…This type of streaming architecture would also be beneficial for solo performances with large audiences. HLS streaming and optimizing the WebRTC audio streams for lower latency [89] are being considered for future development. With the current components, latency-critical NMPs and solo performances are also achievable when using virtual instruments with PdXR or the live coding components, as no audio streaming is required.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Orchestra: A Toolbox for Live Music Performances in a Web-Based Metaverse

Dziwis,

Von Coler,

Pörschmann

2023

J. Audio Eng. Soc.

View full text Add to dashboard Cite

As the potential of networked multiuser virtual environments increases under the concept of the metaverse, so do the interest and artistic possibilities of using them for live music performances. Live performances in online metaverse environments offer an easy and environmentally friendly way to bring together artists and audiences from all over the world. Virtualization also enables countless possibilities for designing and creating artistic experiences and new performance practices. For many years, live performances have been established on various virtual platforms, which differ significantly in terms of possible performance practices, user interaction, immersion, and usability. With Orchestra, we are developing an open-source toolbox that uses the Web Audio Application Programming Interface to realize live performances with various performance practices for web-based metaverse environments. Possibilities vary from live streaming of volumetric audio and video, live coding in multiple (including audiovisual) programming languages, to performing with generative algorithms or virtual instruments developed in Pure Data. These can be combined in various ways and also be used for telematic/networked music ensembles, interactive virtual installations, or novel performance concepts. In this paper, we describe the development and scope of the Orchestra toolbox, as well as use cases that illustrate the artistic possibilities.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Orchestra: A Toolbox for Live Music Performances in a Web-Based Metaverse

Dziwis,

Von Coler,

Pörschmann

2023

J. Audio Eng. Soc.

View full text Add to dashboard Cite

show abstract

“…The reason for such a limitation is that, as far as we know, all these web services were implemented using the Web Audio [16] and WebRTC [17] standards, that do not permit transmitting audio data with low delay and without perceptual audio compression. Even with high bit rates, such compression hinders the audio quality and adds algorithmic delay during the encoding process [18]. In addition, audio processing algorithms designed for voice communications can introduce sound artifacts due to automatic gain adjustment mechanisms that change sound levels.…”

Section: State Of the Artmentioning

confidence: 99%

“…The service architecture leveraged in this paper is an enhanced version of the one presented in [19]. The service is structured around a client component which has two implementations, a web-based one and a native one.…”

Section: Software Architecturementioning

confidence: 99%

Pristine Quality Networked Music Performance System for the Web

Sacchetto,

Severi,

Rottondi

et al. 2024

Proceedings of the 10th Convention of the European Acoustics Association Forum Acusticum 2023

View full text Add to dashboard Cite

The urge for high-speed Internet connectivity, that we have witnessed since the origins of the web, has now have paved the way for pristine quality audio communications, i.e., without artifacts arising from audio compression, and, whenever possible, with low acquisition and processing latency. In this work, we present a software application (part of the HiFiReM project) that provides a unified environment, both native/embedded and web-based, for high-fidelity interactive multi-user musical performances across the Internet. The system is shown to outperform popular video-conferencing applications that adopt stateof-the-art audio compression techniques in terms of perceived audio quality. Subjective quality assessments confirm that, given adequate network conditions, users involved in remote audio communications experience performative conditions close to those of in-presence musical interactions, even if at several miles of distance.

show abstract

“…For the MM case, there is a need for more music-oriented audio streaming. This is particularly challenging for web-based solutions [89]. Moreover, in the MM, there might be scenarios where every note played by a musician needs to be heard by anyone who is connected, or it should be transmitted to a given location at a given time.…”

Section: Lack Of Adequate Audio Toolsmentioning

confidence: 99%

Musical Metaverse: vision, opportunities, and challenges

Turchet

2023

Pers Ubiquit Comput

View full text Add to dashboard Cite

The so-called metaverse relates to a vision of a virtual, digital world which is parallel to the real, physical world, where each user owns and interact through his/her own avatar. Music is one of the possible activities that can be conducted in such a space. The “Musical Metaverse” (MM), the metaverse part which is dedicated to musical activities, is currently in its infancy, although is a concept that is constantly evolving and is progressing at a steady pace. However, to the best of the author’s knowledge, as of today an investigation about the opportunities and challenges posed by the MM has not been conducted yet. In this paper, we provide a vision for the MM and discuss what are the opportunities for musical stakeholders offered by current implementations of the MM, as well as we envision those that are likely to occur as the metaverse emerges. We also identify the technical, artistic, ethical, sustainability, and regulatory issues that need to be addressed so for the MM to be created and utilized in efficient, creative, and responsible ways. Given the importance and timeliness of the MM, we believe that a discussion on the related opportunities and concerns is useful to provide developers with guidelines for creating better virtual environments and musical interactions between stakeholders.

show abstract

Web-Based Networked Music Performances via WebRTC: A Low-Latency PCM Audio Solution

Cited by 9 publications

References 0 publications

Orchestra: A Toolbox for Live Music Performances in a Web-Based Metaverse

Orchestra: A Toolbox for Live Music Performances in a Web-Based Metaverse

Pristine Quality Networked Music Performance System for the Web

Musical Metaverse: vision, opportunities, and challenges

Contact Info

Product

Resources

About