Perceptually Salient Regions of the Modulation Power Spectrum for Musical Instrument Identification

Abstract

The ability of a listener to recognize sound sources, and in particular musical instruments from the sounds they produce, raises the question of determining the acoustical information used to achieve such a task. It is now well known that the shapes of the temporal and spectral envelopes are crucial to the recognition of a musical instrument. More recently, Modulation Power Spectra (MPS) have been shown to be a representation that potentially explains the perception of musical instrument sounds. Nevertheless, the question of which specific regions of this representation characterize a musical instrument is still open. An identification task was applied to two subsets of musical instruments: tuba, trombone, cello, saxophone, and clarinet on the one hand, and marimba, vibraphone, guitar, harp, and viola pizzicato on the other. The sounds were processed with filtered spectrotemporal modulations with 2D Gaussian windows. The most relevant regions of this representation for instrument identification were determined for each instrument and reveal the regions essential for their identification. The method used here is based on a “molecular approach,” the so-called bubbles method. Globally, the instruments were correctly identified and the lower values of spectrotemporal modulations are the most important regions of the MPS for recognizing instruments. Interestingly, instruments that were confused with each other led to non-overlapping regions and were confused when they were filtered in the most salient region of the other instrument. These results suggest that musical instrument timbres are characterized by specific spectrotemporal modulations, information which could contribute to music information retrieval tasks such as automatic source recognition.

Playing Technique Recognition by Joint Time–Frequency Scattering

“…Future work will further verify the approach on other datasets with playing techniques, such as Studio-Online [6] and ConTimbre [18]. We will also compare the jTFST with other equivalent time-frequency representations, such as the two-dimensional Fourier transform and the modulation spectra [19].…”

CLEESE: An open-source audio-transformation toolbox for data-driven experiments in speech and music cognition

“…In the visual domain, these techniques have been extended in recent years to address not only low-level sensory processes, but higher-level cognitive mechanisms in humans: facial recognition [12], emotional expressions [2, 13], social traits [14], as well as their associated individual and cultural variations ([15]; for a review, see [5]). In speech, even more recently, reverse correlation and the associated “bubbles” technique were used to study spectro-temporal regions underlying speech intelligibility [16, 17] or phoneme discrimination in noise [18, 19] and, in music, timbre recognition of musical instruments [20, 21].…”

Specifying the perceptual relevance of onset transients for musical instrument identification

“…From a more general perspective, the current approach is in line with an upsurge of interest in signal analysis/re-synthesis approaches to the study of auditory perception (Mc-Dermott and Simoncelli, 2011;Overath et al, 2015;Ponsot et al, 2018;Thoret et al, 2017).…”

“…Most recently, Thoret et al (2016Thoret et al ( , 2017 showed that instrument identification is determined by specific instrument-specific spectrotemporal modulations, although their approach did not allow them to draw specific conclusions about the role of onsets. Ogg et al (2017) studied the minimal duration required to discriminate between musical instrument sounds, human speech, and human environmental sounds.…”