What Constitutes a Phrase in Sound-Based Music? A Mixed-Methods Investigation of Perception and Acoustics

Abstract

Phrasing facilitates the organization of auditory information and is central to speech and music. Not surprisingly, aspects of changing intensity, rhythm, and pitch are key determinants of musical phrases and their boundaries in instrumental note-based music. Different kinds of speech (such as tone- vs. stress-languages) share these features in different proportions and form an instructive comparison. However, little is known about whether or how musical phrasing is perceived in sound-based music, where the basic musical unit from which a piece is created is commonly non-instrumental continuous sounds, rather than instrumental discontinuous notes. This issue forms the target of the present paper. Twenty participants (17 untrained in music) were presented with six stimuli derived from sound-based music, note-based music, and environmental sound. Their task was to indicate each occurrence of a perceived phrase and qualitatively describe key characteristics of the stimulus associated with each phrase response. It was hypothesized that sound-based music does elicit phrase perception, and that this is primarily associated with temporal changes in intensity and timbre, rather than rhythm and pitch. Results supported this hypothesis. Qualitative analysis of participant descriptions showed that for sound-based music, the majority of perceived phrases were associated with intensity or timbral change. For the note-based piano piece, rhythm was the main theme associated with perceived musical phrasing. We modeled the occurrence in time of perceived musical phrases with recurrent event ‘hazard’ analyses using time-series data representing acoustic predictors associated with intensity, spectral flatness, and rhythmic density. Acoustic intensity and timbre (represented here by spectral flatness) were strong predictors of perceived musical phrasing in sound-based music, and rhythm was only predictive for the piano piece. A further analysis including five additional spectral measures linked to timbre strengthened the models. Overall, results show that even when little of the pitch and rhythm information important for phrasing in note-based music is available, phrasing is still perceived, primarily in response to changes of intensity and timbre. Implications for electroacoustic music composition and music recommender systems are discussed.

Controlling Perception Thresholds for Changing Timbres in Continuous Sounds

“…Traditional approaches have usually focused on simple pairs of rather short sound stimuli, presented in succession, separated by (sometimes not further specified) sound gaps (Caclin, McAdams, Smith and Winsberg 2005; Marozeau and de Cheveigné 2007; Siedenburg et al 2015). However, from a compositional standpoint, questions about perceived changes in sounds with no intervening silence ( gap-free sounds ) are at least as important (Olsen, Dean and Leung 2016). Indeed, most musical phrases consist primarily of gap-free sound, with gaps only indicating the beginning and end of a phrase.…”

“…Assuming that for a change in sound to be noticed an acoustic event has to have taken place, the series of experiments described here assessed how much each of the chosen timbre manipulators (which are detailed in Section 3.3) has to be changed for a participant to notice a new event in a gap-free sound stimulus. An event in this context is here defined to be: a segment of time at a given location that is conceived by an observer to have a beginning and an end (Olsen et al 2016; Zacks, Speer, Swallow, Braver and Reynolds 2007). In a second experiment, we also investigated whether such events can be perceived when the stimuli are gradually transformed from one into another.…”

“…Previous studies have often identified the spectral centroid (SC), the logarithm of attack time (LAT) and, to a lesser extent, spectral flux (SFX) as the most salient correlates of timbre space dimensions (McAdams 1999; McAdams et al 1995). In continuous perceptions of change in music (where features such as intensity and pitch are highly variable), the factors which best model the perceptions are acoustic intensity and other acoustic aspects of timbre, such as spectral flatness, spectral centroid, spectral flux, spectral spread, inharmonicity and roughness (Olsen et al 2016; Olsen, Dean, Stevens and Bailes 2015).…”

Spectral Pitch Similarity is a Predictor of Perceived Change in Sound- as Well as Note-Based Music

“…The pieces ranged from Australian indigenous music to Miles Davis, electroacoustic music, and sound-text, as summarized in Tables 1 and 2. Before commencing the CSTSA, we considered the possible correlation of the eight acoustic features being studied across the music excerpt Dataset 2. Note that, except for SPS, these were already chosen on the basis of prior studies of their relative independence in short sounds (Peeters et al, 2011) and their utility in studies of timbral phrase detection (Olsen et al, 2016). Figure 1 shows the distributions of the measures across 500 ms segments of the whole corpus (treating each segment as an independent sample from the whole set) and the Pearson correlations between them.…”

“…Broadly, these have shown using time series analysis techniques that continuous variation in acoustic intensity and spectral properties (commonly assessed as the MPEG7 high-level parameter spectral flatness) 1 interact as predictors in successful models, with the influence of intensity predominating. Particularly in recent studies of perception of timbres of individual short sounds, or continuous perception of phrasing in timbre-focused music such as some electroacoustic music (Olsen, Dean, & Leung, 2016), a variety of other spectral parameters (e.g., spectral centroid, spectral flux, inharmonicity, roughness, spectral spread) have also been found to be important.…”