Incremental Syllable-Context Phonetic Vocoding

Abstract

Abstract-Current very low bit rate speech coders are, due to complexity limitations, designed to work off-line. This paper investigates incremental speech coding that operates real-time and incrementally (i.e., encoded speech depends only on alreadyuttered speech without the need of future speech information). Since human speech communication is asynchronous (i.e., different information flows being simultaneously processed), we hypothesised that such an incremental speech coder should also operate asynchronously. To accomplish this task, we describe speech coding that reflects the human cortical temporal sampling that packages information into units of different temporal granularity, such as phonemes and syllables, in parallel. More specifically, a phonetic vocoder -cascaded speech recognition and synthesis systems -extended with syllable-based information transmission mechanisms is investigated. There are two main aspects evaluated in this work, the synchronous and asynchronous coding. Synchronous coding refers to the case when the phonetic vocoder and speech generation process depend on the syllable boundaries during encoding and decoding respectively. On the other hand, asynchronous coding refers to the case when the phonetic encoding and speech generation processes are done independently of the syllable boundaries. Our experiments confirmed that the asynchronous incremental speech coding performs better, in terms of intelligibility and overall speech quality, mainly due to better alignment of the segmental and prosodic information. The proposed vocoding operates at an uncompressed bit rate of 213 bits/sec and achieves an average communication delay of 243 ms.

Composition of Deep and Spiking Neural Networks for Very Low Bit Rate Speech Coding

“…Rather, we move the compression one level above, from the acoustics to the phonetic and phonological representation of the speech signal. Our previous work [7] showed that, if such a representation is inferred from the acoustics well, we could decrease the transmission rate to about 200 bit/s, however the speech quality did not achieve the quality of old LPC coders operating at 500 bit/s or less, that is, slightly lower quality than normal LPC coding at 2 kbit/s. In this work, we hypothesise that recent advances in the convergence of deep learning and speech technology could fill the gap, i.e., improving the speech quality of VLBR coding whilst still using a phonetic and phonological speech representation.…”

“…Effective encoding of the F0 signal can be realised by curve fitting done on a syllable level. We thus propose to encode the continuous F0 signal using the discrete (Legendre) orthogonal polynomial (DLOP), as in [7]. To estimate syllable boundaries from the speech signal, a neuromorphic oscillatory device is used, based on modelling brain neural oscillations at syllable frequency.…”

“…The parametric methods known phenomenon; phoneme ASR misrecognition rates tend to increase with longer words [8]. In addition, the coder [7] detected syllable boundaries from recognised phoneme sequences based on the sonority sequencing principle; thus phoneme misrecognition was further propagated to the syllable boundary estimation sometimes resulting in wrongly detected syllables. The HMM coder also used voiced/unvoiced detection for parametrization of the F0 signal in the voiced regions.…”

On structured sparsity of phonological posteriors for linguistic parsing

“…Leong et al (2014) show that phase relations between the phonetic and syllabic amplitude modulations, known as hierarchical phase locking and nesting or synchronization across different temporal granularity (Lakatos et al, 2005), is a good indication of the syllable stress. Intelligible speech representation with stress and accent information can be constructed by asynchronous fusion of phonetic and syllabic information (Cernak et al, 2015a).…”

Neuromorphic based oscillatory device for incremental syllable boundary detection

“…By contrast to conventional syllabification models that work offline, humans "encode" speech in an incremental fashion, i.e., encoded speech does not depend on future temporal context (similar to causality in digital signal processing theory) (Levelt, 1993). We are therefore interested in an incremental syllabification method that can be directly applied to incremental speech processing systems such as in Cernak et al (2015). We hypothesise that a biologically plausible method would fulfill this requirement.…”