A Gestalt inference model for auditory scene segregation

Chakrabarty, Debmalya; Elhilali, Mounya

doi:10.1371/journal.pcbi.1006711

Cited by 19 publications

(10 citation statements)

References 127 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The present model contributes to this effort by showing how adaptation, inhibition and noise may generate auditory bistability within a more comprehensive system. Past efforts that may contribute to our understanding of how brain-like computations can April 3, 2019 12/28 scale to larger systems fall short of this demonstration either because they include comprehensive dynamics of adaptation, inhibition and noise but abstract away from the specific challenges of auditory scene analysis [98][99][100][101] or because they can perform some form of auditory scene analysis but do not show any identified form of adaptation, inhibition and noise nor any bistable output [102][103][104]. Future efforts towards scaling of the present model could build on the strengths of any one of these past systems.…”

Section: Discussionmentioning

confidence: 99%

Ensemble modeling of auditory streaming reveals potential sources of bistability across the perceptual hierarchy

Little

Snyder

Elhilali

2019

Preprint

Self Cite

View full text Add to dashboard Cite

AbstractPerceptual bistability—the spontaneous fluctuation of perception between two interpretations of a stimulus—occurs when observing a large variety of ambiguous stimulus configurations. This phenomenon has the potential to serve as a tool for, among other things, understanding how function varies across individuals due to the large individual differences that manifest during perceptual bistability. Yet it remains difficult to interpret the functional processes at work, without knowing where bistability arises during perception. In this study we explore the hypothesis that bistability originates from multiple sources distributed across the perceptual hierarchy. We develop a hierarchical model of auditory processing comprised of three distinct levels: a Peripheral, tonotopic analysis, a Central analysis computing features found more centrally in the auditory system, and an Object analysis, where sounds are segmented into different streams. We model bistable perception within this system by injecting adaptation, inhibition and noise into one or all of the three levels of the hierarchy. We evaluate a large ensemble of variations of this hierarchical model, where each model has a different configuration of adaptation, inhibition and noise. This approach avoids the assumption that a single configuration must be invoked to explain the data. Each model is evaluated based on its ability to replicate two hallmarks of bistability during auditory streaming: the selectivity of bistability to specific stimulus configurations, and the characteristic log-normal pattern of perceptual switches. Consistent with a distributed origin, a broad range of model parameters across this hierarchy lead to a plausible form of perceptual bistability. The ensemble also appears to predict that greater individual variation in adaptation and inhibition occurs in later stages of perceptual processing.Author summaryOur ability to experience the everyday world through our senses requires that we resolve numerous ambiguities present in the physical evidence available. This is accomplished, in part, through a series of hierarchical computations, in which stimulus interpretations grow increasingly abstract. Our ability to resolve ambiguity does not always succeed, such as during optical illusions. In this study, we examine a form of perceptual ambiguity called bistability—cases in which a single individual’s perception spontaneously switches back and forth between two interpretations of a single stimulus. A challenge in understanding bistability is that we don’t know where along the perceptual hierarchy it is generated. Here we test the idea that there are multiple origins by building a simulation of the auditory system. Consistent with a multi-source account of bistability, this simulation accurately predicts perception of a simple auditory stimulus when bistability originates from a number of different sources within the model. The data also indicate that individual differences during ambiguous perception may primarily originate from higher levels of the perceptual hierarchy. This result provides a clue for future work aiming to determine how auditory function differs across individual brains.

show abstract

Section: Discussionmentioning

confidence: 99%

Ensemble modeling of auditory streaming reveals potential sources of bistability across the perceptual hierarchy

Little

Snyder

Elhilali

2019

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Recent advances in deep neural networks (DNN) and auditory modeling have made it possible to achieve high-levels of sound recognition performance approaching human performance limits and such networks can predict various features of peripheral and central auditory processing [ 50 – 52 ]. Although these networks differ from the proposed HSNN in terms of the computing elements used, the computations performed and the network structure.…”

Section: Discussionmentioning

confidence: 99%

Spiking network optimized for word recognition in noise predicts auditory system hierarchy

Khatami

Escabı́

2020

PLoS Comput Biol

View full text Add to dashboard Cite

The auditory neural code is resilient to acoustic variability and capable of recognizing sounds amongst competing sound sources, yet, the transformations enabling noise robust abilities are largely unknown. We report that a hierarchical spiking neural network (HSNN) optimized to maximize word recognition accuracy in noise and multiple talkers predicts organizational hierarchy of the ascending auditory pathway. Comparisons with data from auditory nerve, midbrain, thalamus and cortex reveals that the optimal HSNN predicts several transformations of the ascending auditory pathway including a sequential loss of temporal resolution and synchronization ability, increasing sparseness, and selectivity. The optimal organizational scheme enhances performance by selectively filtering out noise and fast temporal cues such as voicing periodicity, that are not directly relevant to the word recognition task. An identical network arranged to enable high information transfer fails to predict auditory pathway organization and has substantially poorer performance. Furthermore, conventional single-layer linear and nonlinear receptive field networks that capture the overall feature extraction of the HSNN fail to achieve similar performance. The findings suggest that the auditory pathway hierarchy and its sequential nonlinear feature extraction computations enhance relevant cues while removing non-informative sources of noise, thus enhancing the representation of sounds in noise impoverished conditions.

show abstract

“…Some computational approaches to auditory scene analysis instead frame perceptual organization as clustering "bottom-up" features of audio (99,100). This approach has often been combined with bottom-up features inspired by neurophysiology (11,13,(101)(102)(103), but it remains difficult to identify features whose clustering is comprehensively predictive of perceptual organization. Our approach shares some abstract similarities in combining a bottom-up processing component with a top-down inference stage, but a key difference is that the bottomup component of our analysis-by-synthesis algorithm preserves uncertainty, outputting proposals for event latent variables that can be refined or rejected during inference.…”

Section: Clusteringmentioning

confidence: 99%

Listening with generative models

Cusimano

Hewitt

McDermott

2023

Preprint

View full text Add to dashboard Cite

Perception has long been envisioned to use an internal model of the world to infer the causes of sensory signals. However, tests of inferential accounts of perception have been limited by computational intractability, as inference requires searching through complex hypothesis spaces. Here we revisit the idea of perception as inference in a world model, using auditory scene analysis as a case study. We applied contemporary computational tools to enable Bayesian inference in a structured generative model of auditory scenes. Model inferences accounted for many classic illusions. Unlike most previous accounts of auditory illusions, our model can be evaluated on any sound, and exhibited human-like perceptual organization for real-world sound mixtures. The combination of stimulus-computability and interpretable structure enable 'rich falsification', revealing additional assumptions about sound generation needed to explain perception. The results show how a single generative theory can account for the perception of both classic illusions and everyday sensory signals.

show abstract

A Gestalt inference model for auditory scene segregation

Cited by 19 publications

References 127 publications

Ensemble modeling of auditory streaming reveals potential sources of bistability across the perceptual hierarchy

Ensemble modeling of auditory streaming reveals potential sources of bistability across the perceptual hierarchy

Spiking network optimized for word recognition in noise predicts auditory system hierarchy

Listening with generative models

Contact Info

Product

Resources

About