Modelling Perception of Structure and Affect in Music: Spectral Centroid and Wishart's Red Bird

Abstract

Pearce (2011) provides a positive and interesting response to our article on time series analysis of the influences of acoustic properties on real-time perception of structure and affect in a section of Trevor Wishart's Red Bird (Dean & Bailes, 2010). We address the following topics raised in the response and our paper. First, we analyse in depth the possible influence of spectral centroid, a timbral feature of the acoustic stream distinct from the high level general parameter we used initially, spectral flatness. We find that spectral centroid, like spectral flatness, is not a powerful predictor of real-time responses, though it does show some features that encourage its continued consideration. Second, we discuss further the issue of studying both individual responses, and as in our paper, group averaged responses. We show that a multivariate Vector Autoregression model handles the grand average series quite similarly to those of individual members of our participant groups, and we analyse this in greater detail with a wide range of approaches in work which is in press and continuing. Lastly, we discuss the nature and intent of computational modelling of cognition using acoustic and music-or information theoretic data streams as predictors, and how the music-or information theoretic approaches may be applied to electroacoustic music, which is 'sound-based' rather than note-centred like Western classical music.Submitted 2011 September 30; accepted 2011 October 10.KEYWORDS: time series analysis; musical structure; musical affect; information theoretic analysis; computational modelling of cognition; electroacoustic music. PEARCE (2011) supports our focus on studying real-time responses to music, and appreciates our introducing the methods of time series analysis (which have been rarely used in music studies) and using computer-mediated electroacoustic music as part of this analysis. Indeed, in our ongoing work, we make a point of contrasting and comparing responses to the 'sound-centered' electroacoustic musics, as for example Landy (2009) characterizes them, with those to 'note-centered' music, such as piano music from the Western classical music tradition. Our work allows demonstration not only of the predictive capacity of acoustic information streams for perception of musical structure and affect, but also of interactions between individual perceptual/cognitive responses, and their autoregressive properties.In this article, we respond to three main topics raised by Pearce's comments and by our work to date: the possible influence of the acoustic parameter spectral centroid on perceptions of musical structure and affect; the comparison between group average perceptual responses and individual responses; and finally the nature of computational cognitive modelling and possibilities for developing information theoretic aspects of it in relation to electroacoustic music.

What Constitutes a Phrase in Sound-Based Music? A Mixed-Methods Investigation of Perception and Acoustics

“…This is important because ultimately, sequences of phrases (and recognition of phrase similarities) are the elements of music that contribute to its structure and affective qualities. It is perhaps not surprising (but nevertheless useful) to realize that even in continuous sound environments, spectral flatness and spectral centroid are both important (as found earlier with a sound-based piece; see [20]), together with inharmonicity, roughness, and spectral spread. The relative lack of influence from spectral flux in the present study does not of course rule out its importance, but rather suggests that it is a descriptor whose impact may be driven more directly by component factors such as changes in the individual spectral factors discussed above.…”

Does perceived exertion influence perceived affect in response to music? Investigating the “FEELA” hypothesis.

“…Time-series models of affect included the perceived exertion measure and the “FEELA” hypothesis formed the underlying conceptual framework: the Force and Effort (realized here throughout as physical exertion) required for a performer to produce a musical sound, the Energy of the resulting sound (realized as acoustic intensity), and the experience of the listener in the form of perceived Loudness and Arousal are all hypothesized parts of communication and perception of affect in music (Dean & Bailes, 2010a, 2010b; Dean et al, 2013). Previous time-series models of affect and arousal in particular have reported the significant role of listener engagement (Olsen et al, 2014), perceptual loudness (Olsen et al, 2015), and acoustic intensity (Dean & Bailes, 2010c; Dean et al, 2011). The results from time-series models in the present paper complement and strengthen previous models and show that given reasonably apparent human agency in the production of classical or electroacoustic music, nonmusicians’ perception of exertion required in producing music is pertinent to the perception of arousal and to a lesser extent, valence.…”

“…The BIC was used as the basis of model selection and penalizes strongly for the addition of predictor variables to a model (lowest BIC values are best). Such an approach has been detailed in previous papers (Bailes & Dean, 2012; Dean & Bailes, 2010c, 2011; Dean et al, 2011) and when comparing BIC values between models, if there is an absolute BIC difference (“delta BIC”) of >4.6, the evidence in favor of models with lower BIC is normally described as ‘strong’ and corresponds to a 10-fold difference in probability (Lewandowsky & Farrell, 2011). A delta BIC greater than 1.4 is termed “positive” (a twofold difference in probability), and smaller differences are considered ambiguous as to which model is preferred (Kass & Raftery, 1995).…”

Encouraging Attention and Exploration in a Hybrid Recommender System for Libraries of Unfamiliar Music

“…MPEG-7, the international standard for audio content description under ISO/IEC 15938:2002 (International Organization for Standardization (ISO), 2002) contains seventeen hierarchic spectral and temporal descriptors of music acoustics and instrumental timbres based on perceptual knowledge: such as acoustic intensity, spectral flatness and centroid, log attack time, and brightness (Casey, 2001; Dean & Bailes, 2011). This has led to many applications in MIR (Kim, Moreau, & Sikora, 2006) including audio analysis techniques and machine listening (Jehan, 2005); audio content matching and comparison (Allamanche et al, 2001); automatic classification (Tzanetakis & Cook, 2002); and music recommendation systems (Aggarwal, 2016; Celma, 2010).…”