Instantaneous-frequency distribution vs. time: An interpretation of the phase structure of speech

Abstract

A new time-frequency display is constructed based on the phase of the running short-time Fourier transform, specifically the distribution of its time derivative. Typical results are given for speech, indicating more precise location of formants than is usual for the spectrogram.Some insights as to the pertinent structural aspects of the speech signal can clearly be gained from study of the auditory system in humans and other mammals. The bulk of experimental evidence indicates that the ear performs some sort of running short-time spectral analysis of the acoustic waveform. Recent work has suggested moreover that the information encoded in neural activity consists of more than a representation of spectral amplitude: it appears that phase-locking of individual unit firings to the local basilar-membrane motion is maintained up to several kHz. Recent simulation studies, e.g., [L82,L84] have incorporated this behavior in an effort to determine what characteristics of the signal might be preserved in transmission through the peripheral auditory pathway. There is now increasing justification for the view that the spectral decomposition occurring in the ear serves merely to isolate individual frequencylocalized components of the signal, with further processing done essentially in the time domain. A highlydeveloped instance of such an analysis mode in a mammalian auditory system has recently been pointed out in the case of bat sonar {SBzfl.The implications of this view for artificial speech recognition provide the motivation for this work. One is led to ask whether, in place of the usual time-varying powerspectrum description of the input signal, a better choice for the initial processing step would be one taking into account the phase structure of the speech waveform.This need not involve explicit modeling of the ear, since if the encoding of an acoustic waveform into a neural firing pattern can be understood in terms of an underlying signal-space transformation -in the sense that the displacement of the basilar membrane vs. time can be seen as essentially a running short-time Fourier transform of the acoustic wave -then a variety of conceivable implementations exists from which one may choose that best suited to given means and requirements. Rather than attempt here to deduce such a transformation from first principles and available evidence, we adopt a much more modest goal: to show a new display interpretation of the running short-time Fourier transform (RSTFT) suggestive of, and inspired by. the image of a group of adjacent nerve fibers phaselocked to a single frequency component of the acoustic signal.It is well known that the RSTFT can be interpreted in terms of simultaneous outputs of a bank of band-pass filters with successively-offset center frequencies, all having the given signal waveform as input. If the transform of x(t) is defhied aswhere w(r) is a real-valued, non-negative symmetrical window function, nonzero only for vi C Ff2 (e.g., the Hann window), this can be represented as the output of the tn-th fil...

Further intelligibility results from human listening tests using the short-time phase spectrum

“…Murthy and her colleagues have recently used GDF-based features for ASR (Murthy and Gadde, 2003;Hegde et al, 2004b,c;Alsteris and Paliwal, 2005). The time-derivative of the phase spectrum, most often referred to as the instantaneous frequency distribution (IFD), has been used in the past for pitch extraction (Abe et al, 1995;Charpentier, 1986;Nakatani et al, 2003) and formant extraction (Potamianos and Maragos, 1996;Friedman, 1985). Potamianos and Maragos (2001), Dimitriadis and Maragos (2003), Paliwal and Atal (2003) and Wang et al (2003) have recently investigated IFD-based features for ASR.…”

Algorithms for computing the time-corrected instantaneous frequency (reassigned) spectrogram, with applications

“…These two methods are also provided with step-by-step algorithms here, and their performance relative to the naïve benchmark is anecdotally and mathematically evaluated. A third independent method was also published, 15 but this seems to be a partial foreshadowing of the method of Auger and Flandrin, 3 and so will not be separately treated here. A host of more distantly related work on increasing precision in time-frequency analysis has also been published over the years 16 but also will not be considered because the contributions do not directly modify the spectrogram as closely as our main subject, the TCIF spectrogram.…”

Time-Frequency Reassignment: From Principles to Algorithms

“…Algorithmic improvements were proposed and, thanks to the many developments that occurred the field, the scope of the technique was considerably enlarged, far beyond only the spectrogram case. In parallel, other related techniques were developed independently(e.g., the "ridge and skeleton" method [5,13,19,20], the "instantaneous frequency density" [17], the "differential spectral analysis" [18] or the "synchrosqueezing" technique [29]).…”