Feature-Based Analysis of the Effects of Packet Delay on Networked Musical Interactions

Abstract

Networked Music Performance (NMP) is a mediated interactional modality with a tremendous potential impact on professional and amateur musicians, as it enables real-time interaction from remote locations. One of the known limiting factors of distributed networked performances is the impact of the unavoidable packet delay and jitter introduced by IP networks, which make it difficult to keep a stable tempo during the performance. This paper investigates the tolerance of remotely interacting musicians towards adverse network conditions. We do so for various musical instruments and music genres, as a function of rhythmic complexity and tempo. In order to conduct this analysis, we implemented a testbed for psycho-acoustic analysis emulating the behavior of a real IP network in terms of variable transmission delay and jitter, and we quantitatively evaluated the impact of such parameters on the trend of the tempo maintained during the performance and on the perceptual quality of the musical interaction.

A Presence- and Performance-Driven Framework to Investigate Interactive Networked Music Learning Scenarios

“…e resulting latency level represents a factor that dramatically affects the NMP experience. It has been shown that latency values in the range of 20-60 ms degrade the quality of the performance, by causing progressive slowdowns in the performance [19]. e effect of unnatural latencies has been explained in terms of incongruent temporal separation between action and auditory information, with respect to the expected "natural" time delays occurring in co-located performances [16,17,34].…”

“…e concert is the con guration commonly investigated in NMP research and technological development [4,5,59]. Multimodal signal acquisition includes the audio recordings from both the remote environments, to compute several measures on the quality of the performance [19], gaze tracing and eye tracking to annotate the visual search of the musicians [60], and the biometric response of the performers for distress estimation [61].…”

“…In general, the user types considered in the InterMUSIC scenario are musicians with at least 5 years of academical music practice. e instrument is a relevant feature of the subject, and content-based analyses are used to integrate the user's profile [19]. Previous experiences with virtual environments and motivation are also relevant individual information to account an attitude to adaptability and selective attention.…”

“…In particular, we approach the issue of synchronicity in NMP as a necessary condition for effective musical interaction, by focusing on how rhythm, melody, and expression affect the interaction and communication between performers. e existing literature on NMP pinpointed several factors affecting the objective quality of the music performance, namely, the unavoidable network latencies [16] and the effect of the resulting temporal separation of action and feedback [17], the timbral properties of the instruments [18], and the rhythmic complexity of the performed pieces [19]. ese constitute the point of departure towards an interaction-centered and situated research approach to the case of learning and practicing chamber music in the NMP environment [20].…”

“…e exercise is specifically conceived around musical structures which are functional to pinpointing a set of problems relative to time management, communication mechanisms, and mutual understanding between remote performers. e co-representation and coordination of duos are investigated by means of a presence questionnaire [24] and combined with objective metrics of the quality of the performance [19]. In this respect, we are exploring how and whether simulated network delay affects the subjective experience of being simultaneously present in the shared reality environment [25].…”