Acoustic Correlates of Auditory Object and Event Perception: Speakers, Musical Timbres, and Environmental Sounds

Abstract

Human listeners must identify and orient themselves to auditory objects and events in their environment. What acoustic features support a listener’s ability to differentiate the great variety of natural sounds they might encounter? Studies of auditory object perception typically examine identification (and confusion) responses or dissimilarity ratings between pairs of objects and events. However, the majority of this prior work has been conducted within single categories of sound. This separation has precluded a broader understanding of the general acoustic attributes that govern auditory object and event perception within and across different behaviorally relevant sound classes. The present experiments take a broader approach by examining multiple categories of sound relative to one another. This approach bridges critical gaps in the literature and allows us to identify (and assess the relative importance of) features that are useful for distinguishing sounds within, between and across behaviorally relevant sound categories. To do this, we conducted behavioral sound identification (Experiment 1) and dissimilarity rating (Experiment 2) studies using a broad set of stimuli that leveraged the acoustic variability within and between different sound categories via a diverse set of 36 sound tokens (12 utterances from different speakers, 12 instrument timbres, and 12 everyday objects from a typical human environment). Multidimensional scaling solutions as well as analyses of item-pair-level responses as a function of different acoustic qualities were used to understand what acoustic features informed participants’ responses. In addition to the spectral and temporal envelope qualities noted in previous work, listeners’ dissimilarity ratings were associated with spectrotemporal variability and aperiodicity. Subsets of these features (along with fundamental frequency variability) were also useful for making specific within or between sound category judgments. Dissimilarity ratings largely paralleled sound identification performance, however the results of these tasks did not completely mirror one another. In addition, musical training was related to improved sound identification performance.

The Rapid Emergence of Auditory Object Representations in Cortex Reflect Central Acoustic Attributes

“…Exploring the mechanisms of sound source decoding further, our results suggest that the auditory system prioritizes cues related to a sound's spectrotemporal variability and aperiodicity as well as its spectral and temporal envelopes (at least among this set of sounds). These features can effectively distinguish instrument and human-environmental sounds (especially early after onset), which occupy opposing extremes on these dimensions (Ogg et al, 2017;Ogg & Slevc, 2019a). Speech utterances, on the other hand (an especially important sound category for humans), exploit all of these acoustic cues and their extremes from moment to moment to relay linguistic information (Smith & Lewicki, 2006;Stilp & Kluender, 2010).…”

“…Speech tokens were 12 excerpts of the utterances "bead," "bod," "heed," and "hod" as well as an excerpted /ɑ/ and /i/. These were spoken by a male and female raised in the Mid-Atlantic United States and recorded in-house (under conditions described in Ogg et al, 2017 andSlevc, 2019a). These consonants were selected to match the variability of the other sound categories in terms of their noisiness and temporal envelope dynamics.…”

“…Stimuli were edited to begin at their onsets, defined here as 5-ms prior to where the absolute value of the stimulus' amplitude reached 10% of its maximum. All stimuli were then truncated to a 300-ms duration, which is sufficient to facilitate sound source identification following onset (Ogg et al, 2017;Ogg & Slevc, 2019a;Robinson & Patterson, 1995;Suied et al, 2014). Cosine onset and offset ramps (both 5-ms) were then applied.…”

“…To ensure the acoustic analyses were the most accurate representation of the stimuli heard in the scanner, acoustic features were derived from high fidelity recordings of each stimulus from the scanner's playback system. From these recordings we extracted acoustic features known to influence the neural processing of sound in humans (Alluri et al, 2012;Giordano et al, 2012;Allen et al, 2017;Lewis et al, 2012;Warren et al, 2005;Lewis et al, 2009;Patterson et al, 2002;Ogg & Slevc, 2019a;2019bMenon et al, 2002Langers et al, 2007;Norman-Haignere et al, 2015;Schönwiesner & Zatorre, 2009) Temporal dynamics of the stimuli (regarding the speed of sound onset) were characterized via the log-attack-time and temporal centroid of the energy envelope (Peeters et al, 2011, Kazazis et al, 2017. Spectral qualities were assessed via spectral centroid (similar to overall brightness), spectral flatness (similar to overall noisiness of the spectrum) and spectral variability (index of spectral change over time) extracted from an ERB (equivalent rectangular-bandwidth) gammatone representation (i.e., a cochleagram) of the sounds (Peeters et al, 2011, Kazazis et al, 2017 in 5-ms windows/increments.…”

“…However, there are many inherent (and important) acoustic differences that exist between different categories of sound (or even among sound sources within the same category) that could drive these results (Ogg & Slevc, 2019a), which have not been explicitly analyzed in previous M/EEG studies of natural sounds. Additionally, work on classic auditory evoked responses (using simple synthesized stimuli) such as the mismatch negativity (Caclin et al, 2006;Giard et al, 1995;Rosburg, 2003), N1 (Näätänen & Picton, 1987) and M100 response (Roberts, Ferrari, Stufflebeam, & Poeppel, 2000) indicate that the auditory system is sensitive to specific acoustic qualities (e.g., spectral and temporal envelopes, noisiness, spectral variability).…”