Deep neural network models reveal interplay of peripheral coding and stimulus statistics in pitch perception

Abstract

Perception is thought to be shaped by the environments for which organisms are optimized. These influences are difficult to test in biological organisms but may be revealed by machine perceptual systems optimized under different conditions. We investigated environmental and physiological influences on pitch perception, whose properties are commonly linked to peripheral neural coding limits. We first trained artificial neural networks to estimate fundamental frequency from biologically faithful cochlear representations of natural sounds. The best-performing networks replicated many characteristics of human pitch judgments. To probe the origins of these characteristics, we then optimized networks given altered cochleae or sound statistics. Human-like behavior emerged only when cochleae had high temporal fidelity and when models were optimized for naturalistic sounds. The results suggest pitch perception is critically shaped by the constraints of natural environments in addition to those of the cochlea, illustrating the use of artificial neural networks to reveal underpinnings of behavior.

Human discrimination and modeling of high-frequency complex tones shed light on the neural codes for pitch

“…Further research integrating psychophysical and physiological data via computational modeling will be needed to understand how limitations originating in non-peripheral loci contribute to PLOS COMPUTATIONAL BIOLOGY complex pitch perception in humans. A wide range of modeling frameworks, including idealobserver analysis [27,28], template-based models [64,65], and neural-network models [71], may all contribute to this endeavor in meaningful ways. For example, whereas neural-network models excel at shedding light on the links between statistical patterns in natural stimuli and complex pitch perception, ideal-observer models can help understand tasks (such as the present high-F0 pitch discrimination) that employ stimuli with low prevalence among natural sounds (i.e., stimuli that are sparsely represented in the naturalistic training data needed for deep neural networks).…”

“…As observed previously for noise maskers [69], interactions PLOS COMPUTATIONAL BIOLOGY between targets and maskers that would never be usable by human observers (because they are completely unpredictable from trial to trial) can be exploited by the ideal observer, leading to unrealistic thresholds. To resolve these limitations of the present ideal-observer model, future work could explore the application of quasi-ideal observers [69,70], deep neural network models [71], or other modeling frameworks to our stimuli to better understand how F0 cues may be more realistically decoded in HCT-mixture stimuli.…”

Periodicity Pitch Perception Part III: Sensibility and Pachinko Volatility

“…Oscillations of octopus neurons are perceived by IC neurons as differentiable pitch sensations. Sounds are transformed into spike-based event representations by a bio-plausible, neuro-physiologically parameterized auditory model (Harczos, 2015 ; James et al, 2017 ; Cramer et al, 2020 ; Gutkin, 2020 ; Baby et al, 2021 ; Gutierrez-Galan et al, 2021 ; Saddler et al, 2021 ). First, the auditory model computes spike train patterns for auditory nerve fibers (ANFs).…”

Multivariate analysis of brain activity patterns as a tool to understand predictive processes in speech perception

“…Technical advances with ultra-high field fMRI have facilitated neuronal layer-specific analysis which could disentangle predictive bottom-up and top-down processes in the visual and somatosensory domains and has yet to be applied to auditory processing (Kok et al, 2016;Muckli et al, 2015;Yu et al, 2019). In addition, recent work demonstrated how neural networks are able to reproduce human auditory behavioural patterns when trained on human sounds, suggesting that the underlying mechanisms may converge between neural networks and human neural representations (Kell et al, 2018;Saddler et al, 2021). Indeed, it has been found that the better the performance of a neural network, the better it mapped onto neuronal data (Schrimpf et al, 2021).…”