Human-Like Modulation Sensitivity Emerging through Optimization to Natural Sound Recognition

Koumura, Takuya; Terashima, Hiroki; Furukawa, Shigeto

doi:10.1523/jneurosci.2002-22.2023

Cited by 7 publications

(3 citation statements)

References 86 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another study optimized a network for a small-scale (10-digit) word recognition task and reported seeing some neurophysiological properties of the auditory system [38]. Koumura and colleagues [37,107] trained networks on environmental sound or speech classification and observed tuning to amplitude modulation, similar to that found in peripheral and mid-level stages of biological auditory systems but did not investigate the putative hierarchy of cortical regions. Giordano and colleagues [44] found that 3 DNN models predicted non-primary voxel responses better than standard acoustic features, generally consistent with the results shown here.…”

Section: Plos Biologymentioning

confidence: 97%

Many but not all deep neural network audio models capture brain responses and exhibit correspondence between model stages and brain regions

Tuckute,

Feather,

Boebinger

et al. 2023

PLoS Biol

View full text Add to dashboard Cite

Models that predict brain responses to stimuli provide one measure of understanding of a sensory system and have many potential applications in science and engineering. Deep artificial neural networks have emerged as the leading such predictive models of the visual system but are less explored in audition. Prior work provided examples of audio-trained neural networks that produced good predictions of auditory cortical fMRI responses and exhibited correspondence between model stages and brain regions, but left it unclear whether these results generalize to other neural network models and, thus, how to further improve models in this domain. We evaluated model-brain correspondence for publicly available audio neural network models along with in-house models trained on 4 different tasks. Most tested models outpredicted standard spectromporal filter-bank models of auditory cortex and exhibited systematic model-brain correspondence: Middle stages best predicted primary auditory cortex, while deep stages best predicted non-primary cortex. However, some state-of-the-art models produced substantially worse brain predictions. Models trained to recognize speech in background noise produced better brain predictions than models trained to recognize speech in quiet, potentially because hearing in noise imposes constraints on biological auditory representations. The training task influenced the prediction quality for specific cortical tuning properties, with best overall predictions resulting from models trained on multiple tasks. The results generally support the promise of deep neural networks as models of audition, though they also indicate that current models do not explain auditory cortical responses in their entirety.

show abstract

Section: Plos Biologymentioning

confidence: 97%

Many but not all deep neural network audio models capture brain responses and exhibit correspondence between model stages and brain regions

Tuckute,

Feather,

Boebinger

et al. 2023

PLoS Biol

View full text Add to dashboard Cite

show abstract

“…All natural sounds such as music and speech can be described in the form of amplitude modulations over time and frequency (Singh & Theunissen, 2003). These modulations are crucial for speech perception (Chi et al, 1999), and the auditory system appears to be highly selective towards and specialised for amplitude-modulated natural sounds such as speech (Joris, Schreiner, & Rees, 2004;Koumura, Terashima, & Furukawa, 2023;Liang, Lu, & Wang, 2002;Yin et al, 2011). The system is also robust to degraded spectral resolution (Remez et al, 1981) something which often occurs in hearing-impaired listeners or users of cochlear implantsso long as amplitude modulations are preserved.…”

Section: From Sound To Perceptionmentioning

confidence: 99%

Phonetics in the Brain

Söderström

2024

View full text Add to dashboard Cite

Spoken language is a rapidly unfolding signal: a complex code that the listener must crack to understand what is being said. From the structures of the inner ear through to higher-order areas of the brain, a hierarchy of interlinked processes transforms the acoustic signal into a linguistic message within fractions of a second. This Element outlines how speech is perceived and explores what the auditory system needs to achieve to make this possible. It traces a path through the system and discusses the mechanisms that enable us to perceive speech as a coherent sequence of words. This is combined with a brief history of research into language and the brain beginning in the nineteenth century, as well as an overview of the state-of-the-art neuroimaging and analysis techniques that are used to investigate phonetics in the brain today. This title is also available as Open Access on Cambridge Core.

show abstract

“…This model-driven approach offers a unique opportunity to map in greater depth the information-processing architecture of the human auditory system in the case of natural sounds and test for the existence of central mechanisms selectively tuned for biological sound-source detection and biodiversity assessment. This approach already proved to be successful for speech and music perception (e.g., Kell et al, 2018;Koumura et al, 2019Koumura et al, , 2023Saddler et al, 2021).…”

Section: Direction II Experimental and Computational Paradigms For St...mentioning

confidence: 99%

Human Auditory Ecology: Extending Hearing Research to the Perception of Natural Soundscapes by Humans in Rapidly Changing Environments

Lorenzi,

Apoux,

Grinfeder

et al. 2023

Trends in Hearing

View full text Add to dashboard Cite

Research in hearing sciences has provided extensive knowledge about how the human auditory system processes speech and assists communication. In contrast, little is known about how this system processes “natural soundscapes,” that is the complex arrangements of biological and geophysical sounds shaped by sound propagation through non-anthropogenic habitats [Grinfeder et al. (2022). Frontiers in Ecology and Evolution. 10: 894232]. This is surprising given that, for many species, the capacity to process natural soundscapes determines survival and reproduction through the ability to represent and monitor the immediate environment. Here we propose a framework to encourage research programmes in the field of “human auditory ecology,” focusing on the study of human auditory perception of ecological processes at work in natural habitats. Based on large acoustic databases with high ecological validity, these programmes should investigate the extent to which this presumably ancestral monitoring function of the human auditory system is adapted to specific information conveyed by natural soundscapes, whether it operate throughout the life span or whether it emerges through individual learning or cultural transmission. Beyond fundamental knowledge of human hearing, these programmes should yield a better understanding of how normal-hearing and hearing-impaired listeners monitor rural and city green and blue spaces and benefit from them, and whether rehabilitation devices (hearing aids and cochlear implants) restore natural soundscape perception and emotional responses back to normal. Importantly, they should also reveal whether and how humans hear the rapid changes in the environment brought about by human activity.

show abstract

Human-Like Modulation Sensitivity Emerging through Optimization to Natural Sound Recognition

Cited by 7 publications

References 86 publications

Many but not all deep neural network audio models capture brain responses and exhibit correspondence between model stages and brain regions

Many but not all deep neural network audio models capture brain responses and exhibit correspondence between model stages and brain regions

Phonetics in the Brain

Human Auditory Ecology: Extending Hearing Research to the Perception of Natural Soundscapes by Humans in Rapidly Changing Environments

Contact Info

Product

Resources

About