Statistics of natural reverberation enable perceptual separation of sound and space

Abstract

In everyday listening, sound reaches our ears directly from a source as well as indirectly via reflections known as reverberation. Reverberation profoundly distorts the sound from a source, yet humans can both identify sound sources and distinguish environments from the resulting sound, via mechanisms that remain unclear. The core computational challenge is that the acoustic signatures of the source and environment are combined in a single signal received by the ear. Here we ask whether our recognition of sound sources and spaces reflects an ability to separate their effects and whether any such separation is enabled by statistical regularities of real-world reverberation. To first determine whether such statistical regularities exist, we measured impulse responses (IRs) of 271 spaces sampled from the distribution encountered by humans during daily life. The sampled spaces were diverse, but their IRs were tightly constrained, exhibiting exponential decay at frequency-dependent rates: Mid frequencies reverberated longest whereas higher and lower frequencies decayed more rapidly, presumably due to absorptive properties of materials and air. To test whether humans leverage these regularities, we manipulated IR decay characteristics in simulated reverberant audio. Listeners could discriminate sound sources and environments from these signals, but their abilities degraded when reverberation characteristics deviated from those of real-world environments. Subjectively, atypical IRs were mistaken for sound sources. The results suggest the brain separates sound into contributions from the source and the environment, constrained by a prior on natural reverberation. This separation process may contribute to robust recognition while providing information about spaces around us.natural scene statistics | auditory scene analysis | environmental acoustics | psychophysics | psychoacoustics P erception requires the brain to determine the structure of the world from the energy that impinges upon our sensory receptors. One challenge is that most perceptual problems are illposed: The information we seek about the world is underdetermined given the sensory input. Sometimes this is because noise partially obscures the structure of interest. In other cases, it is because the sensory signal is influenced by multiple causal factors in the world. In vision, the light that enters the eye is a function of the surface pigmentation we typically need to estimate, but also of the illumination level. In touch, estimates of surface texture from vibrations are confounded by the speed with which a surface passes over the skin's receptors. And in hearing, we seek to understand the content of individual sound sources in the world, but the ear often receives a signal that is a mixture of multiple sources. These problems are all examples of scene analysis, in which the brain must infer one or more of the multiple factors that created the signals it receives (1). Inference in such cases is possible only with the aid of prior information about the world...

Hearing Scenes: A Neuromagnetic Signature of Auditory Source and Reverberant Space Separation

“…Still, a source/reverberant space separation operation by the auditory system would facilitate computing DRR for distance perception. Our data do not address this question directly but, along with Traer and McDermott (2016), suggest an intriguing counterpoint to interpretations that DRR computation is ill-posed and thus unlikely (Kopco and Shinn-Cunningham, 2011), or is bypassed via perception of other covarying cues (Larsen et al, 2008). …”

“…Recent behavioral work suggests that the auditory system performs a scene analysis operation in which natural reverberation is separated from the originating sound source and analyzed to extract environmental information (Traer and McDermott, 2016). The neural basis of that operation, however, remains largely unexplored.…”

Ecological origins of perceptual grouping principles in the auditory system

“…Although our methodology starts from an encoding scheme based on local features, in part because these are most readily mapped onto early stages of sensory systems [64,65,66], problems of scene analysis can also be approached with generative models more rooted in how sounds are produced. For instance, speech and instrument sounds are fruitfully characterized as the product of a source and a filter that each vary over time in particular ways [67,68], as are sounds in reverberant environments [69], and humans appear to have implicit knowledge of this generative structure [70]. Reconciling these generative models for sound with those rooted in neurally plausible local feature decompositions is a critical topic for future research.…”

Auditory brainstem models: adapting cochlear nuclei improve spatial encoding by the medial superior olive in reverberation

“…200 Hz, where listeners are equally sensitive to ITDs conveyed during rising and peak energy. 98 Acoustically, reverberant energy can be 20 dB less intense at 200 Hz than at 600 Hz in many 99 outdoor settings (Traer & McDermott, 2016). For both frequencies, listeners are least sensitive 100…”

“…Frequency-dependent emphasis of early-arriving sound reflects natural 688 frequency-profiles in reverberant energy 689 Natural outdoor acoustics have seemingly influenced brain mechanisms that suppress 690 responses to reverberation. In many outdoor environments, including forests, fields, and 691 streets, reverberation-time ('T60'-the time for reverberant energy to fall by 60 decibels) 692 decreases as sound-frequency decreases below 1500 Hz (Traer & McDermott, 2016). 693…”