A unitary model of auditory frequency change perception

Abstract

Changes in the frequency content of sounds over time are arguably the most basic form of information about the behavior of sound-emitting objects. In perceptual studies, such changes have mostly been investigated separately, as aspects of either pitch or timbre. Here, we propose a unitary account of “up” and “down” subjective judgments of frequency change, based on a model combining auditory correlates of acoustic cues in a sound-specific and listener-specific manner. To do so, we introduce a generalized version of so-called Shepard tones, allowing symmetric manipulations of spectral information on a fine scale, usually associated to pitch (spectral fine structure, SFS), and on a coarse scale, usually associated timbre (spectral envelope, SE). In a series of behavioral experiments, listeners reported “up” or “down” shifts across pairs of generalized Shepard tones that differed in SFS, in SE, or in both. We observed the classic properties of Shepard tones for either SFS or SE shifts: subjective judgements followed the smallest log-frequency change direction, with cases of ambiguity and circularity. Interestingly, when both SFS and SE changes were applied concurrently (synergistically or antagonistically), we observed a trade-off between cues. Listeners were encouraged to report when they perceived “both” directions of change concurrently, but this rarely happened, suggesting a unitary percept. A computational model could accurately fit the behavioral data by combining different cues reflecting frequency changes after auditory filtering. The model revealed that cue weighting depended on the nature of the sound. When presented with harmonic sounds, listeners put more weight on SFS-related cues, whereas inharmonic sounds led to more weight on SE-related cues. Moreover, these stimulus-based factors were modulated by inter-individual differences, revealing variability across listeners in the detailed recipe for “up” and “down” judgments. We argue that frequency changes are tracked perceptually via the adaptive combination of a diverse set of cues, in a manner that is in fact similar to the derivation of other basic auditory dimensions such as spatial location.

Relative pitch representations and invariance to timbre

“…We found that discrimination was only partially invariant to real-world differences in spectral envelope between different vowels or instruments. Judgments were biased depending on whether the spectral envelope shifted congruently or incongruently with the f0, consistent with previous studies (Allen & Oxenham, 2014; Chuang & Wang, 1978; Hellström et al, 1994; Kuhl & Miller, 1982; Micheyl & Oxenham, 2004; Miller, 1978; Moore & Glasberg, 1990; Repp & Lin, 1990; Siedenburg et al, 2022; Singh & Hirsh, 1992; Stoll, 1984). However, invariance was similarly limited for harmonic and inharmonic tones, indicating that invariance does not depend on representations of f0 (addressing the first question).…”

“…We built on previous studies examining pitch discrimination between tones differing in spectral envelope (Allen & Oxenham, 2014; Chuang & Wang, 1978; Hellström et al, 1994; Kuhl & Miller, 1982; Micheyl & Oxenham, 2004; Miller, 1978; Moore et al, 1992; Moore & Glasberg, 1990; Repp & Lin, 1990; Russo & Thompson, 2005; Siedenburg et al, 2022; Singh & Hirsh, 1992; Stoll, 1984; Warrier & Zatorre, 2002), by using comparisons between harmonic and inharmonic stimuli to test whether invariances in pitch perception are dependent on f0-based pitch. In both Experiment 1 (speech) and Experiment 2 (musical instruments), participants showed above-chance up-down discrimination irrespective of whether the instruments or speakers being compared were the same or different.…”

In-channel cancellation: a model of early auditory processing

“…It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for this this version posted December 1, 2022. ; https://doi.org/10. 1101/2022 to in-channel cancellation. In the harmonic cancellation model, all peripheral channels are filtered by a cancellation filter with the same delay parameter 𝑇 , whereas here 𝑇 is set independently in each channel.…”

“…It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for this this version posted December 1, 2022. ; https://doi.org/10. 1101/2022 The in-channel cancellation model can be invoked to support an array of in-channel frequency shift detectors, for example as schematized in Fig. 14.…”

“…CC-BY-NC-ND 4.0 International license perpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for this this version posted December 1, 2022. ; https://doi.org/10.1101/2022…”