Auditory salience using natural soundscapes

Abstract

Salience describes the phenomenon by which an object stands out from a scene. While its underlying processes are extensively studied in vision, mechanisms of auditory salience remain largely unknown. Previous studies have used well-controlled auditory scenes to shed light on some of the acoustic attributes that drive the salience of sound events. Unfortunately, the use of constrained stimuli in addition to a lack of well-established benchmarks of salience judgments hampers the development of comprehensive theories of sensory-driven auditory attention. The present study explores auditory salience in a set of dynamic natural scenes. A behavioral measure of salience is collected by having human volunteers listen to two concurrent scenes and indicate continuously which one attracts their attention. By using natural scenes, the study takes a data-driven rather than experimenter-driven approach to exploring the parameters of auditory salience. The findings indicate that the space of auditory salience is multidimensional (spanning loudness, pitch, spectral shape, as well as other acoustic attributes), nonlinear and highly context-dependent. Importantly, the results indicate that contextual information about the entire scene over both short and long scales needs to be considered in order to properly account for perceptual judgments of salience.

Rapid ocular responses are a robust marker for bottom-up driven auditory salience

“…The link between roughness and subjective salience is in line with previous reports (e.g. Arnal et al, 2015;Huang and Elhilali, 2017;Sato et al, 2007) (Itti and Koch, 2001). We found that none of the parameters derived from that model correlated with the present crowd-sourced scale.…”

“…We found that, despite the fact that loudness is known to be a prominent contributor to perceptual salience (Huang and Elhilali, 2017;Kayser et al, 2005;Liao et al, 2016), subjective salience in the present set did not correlate with loudness (p=0.076; see Methods for details about the loudness measure). This may be partly because level was RMSnormalised thus removing some of the larger differences in loudness between sounds.…”

“…Motivated by previous work (Arnal et al, 2015;Huang and Elhilali, 2017;Sato et al, 2007), we also investigated the correlation between perceptual salience and roughnessoften quantified by taking the energy in the high end (>30Hz) of the amplitude modulation spectrum (e.g. Arnal et al, 2015).…”

Sensitivity to statistical structure facilitates perceptual analysis of complex auditory scenes

“…Transients (onsets of individual tones) in regular streams are predictable and thus do not require substantial processing resources, making it easier to ignore regular patterns when these are task irrelevant (Andreou et al, 2011;Southwell et al, 2017). On the other hand, unexpected events in an otherwise predictable stream are rendered as 'surprising' or 'salient' and capture bottom-up attention (Kaya & Elhilal, 2014;Huang & Elhilali, 2017;Southwell & Chait, 2018).…”

EEG-based classification of natural sounds reveals specialized responses to speech and music

“…is still an open question as to whether the selectivity we observe here is a result of "bottom-up" 447 processing, due to auditory processing that does not require subject engagement, or "top-down" 448 19 factors that depend upon the subject's cognitive state. Disambiguating these two is tricky 449 because specific stimuli with particular acoustic characteristics such as sparsity and roughness 450 could engage a subject's attention (Huang and Elhilali, 2017;Zhao et al, 2019). However, we 451 think it is unlikely that subjects were actively attending to speech and music more than other 452 sounds in the stimulus set because the behavioral task places no special emphasis on these 453 stimuli and all stimuli likely became less interesting over many repeats (>80 presentations).…”