Deep neural network models reveal interplay of peripheral coding and stimulus statistics in pitch perception

Abstract

Computations on receptor responses enable behavior in the environment. Behavior is plausibly shaped by both the sensory receptors and the environments for which organisms are optimized, but their roles are often opaque. One classic example is pitch perception, whose properties are commonly linked to peripheral neural coding limits rather than environmental acoustic constraints. We trained artificial neural networks to estimate fundamental frequency from simulated cochlear representations of natural sounds. The best-performing networks replicated many characteristics of human pitch judgments. To probe how our ears and environment shape these characteristics, we optimized networks given altered cochleae or sound statistics. Human-like behavior emerged only when cochleae had high temporal fidelity and when models were optimized for natural sounds. The results suggest pitch perception is critically shaped by the constraints of natural environments in addition to those of the cochlea, illustrating the use of contemporary neural networks to reveal underpinnings of behavior.

Deep neural network models of sound localization reveal how perception is adapted to real-world environments

“…The analogies with the brain thus seem most promising at the level of behavior and representations. Our results add to growing evidence that task-optimized models can produce human-like behavior for signals that are close to the manifold of natural sounds or images 50 , 116 , 117 . However, artificial neural networks also often exhibit substantial representational differences with humans, particularly for unnatural signals derived in various ways from a network 118 – 122 , and our model may exhibit similar divergences.…”

Harmonic Cancellation—A Fundamental of Auditory Scene Analysis

“…This is non-trivial: the dictionary of templates must cover the full range of F 0s, there must be some mechanism to align the templates accurately with the substrate of frequency analysis (e.g., cochlea), and each template itself is a complex affair involving multiple slots with accurate tuning. It has been proposed that templates are learned from exposure to harmonic sounds such as speech ( Terhardt , 1974 ; Divenyi , 1979 ; Bowling & Purves , 2015 ; Saddler et al , 2020 ) possibly modulated by cultural preferences ( McDermott & Hauser , 2004 ; McDermott et al , 2010 , 2016 ; McPherson et al , 2020 ). The demonstration that templates can be learned from noise ( Shamma & Klein , 2000 ; Shamma & Dutta , 2019 ) makes that argument more tenuous, and highlights the question of what, exactly, is being learned.…”

Harmonic Cancellation - a Fundamental of Auditory Scene Analysis

“…cochlea), and each template itself is a complex affair involving multiple slots with accurate tuning. It has been proposed that templates are learned from exposure to harmonic sounds such as speech (Terhardt, 1974;Divenyi, 1979;Bowling and Purves, 2015;Saddler et al, 2020) possibly modulated by cultural preferences (Mcdermott and Hauser, 2004;McDermott et al, 2010McDermott et al, , 2016. The demonstration that templates can be learned from noise (Shamma and Klein, 2000;Shamma and Dutta, 2019) makes that argument more tenuous, but it then begs the question as to what is being learned.…”