Neuromorphic based oscillatory device for incremental syllable boundary detection

Abstract

Syllables are considered as basic supra-segmental units, used mainly in prosodic modelling. It has long been thought that efficient syllabification algorithms may also provide valuable cues for improved segmental (acoustic) modelling. However, the best current syllabification methods work offline, considering the power envelope of whole utterance.In this paper we introduce a new method for detection of syllable boundaries based on a model of speech parsing into syllables by neural oscillations in human auditory cortex. Neural oscillations automatically lock to speech slow fluctuations that convey the syllabic rhythm. Similarly as humans encode speech incrementally, i.e., not considering future temporal context, the proposed method works incrementally as well. In addition, it is highly robust to noise. Syllabification performance for English and different noise conditions was compared to the existing Mermelstein and group delay algorithms. While the performance of the existing methods depend on the type of noise and signal to noise ratio, the performance of the proposed method is constant under all noise conditions.

Rhythmic modulation of prediction errors: a possible role for the beta-range in speech processing

“…Precoss-β was built by including oscillating state-dependent precisions in a previous generative model version (Precoss) (Hovsepyan, Olasagasti and Giraud, 2020). The model input consists of a speech reduced auditory spectrogram (Chi, Ru and Shamma, 2005) and of its slow amplitude modulations (Hyafil et al, 2015), both extracted English sentences of the TIMIT database (Garofolo et al, 1993) (see Hovespyan et al 2020 (Hovsepyan, Olasagasti and for details about speech input generation). In Precoss-β, the activation of the appropriate syllable unit generates the corresponding auditory spectrogram with a flexible duration determined by eight gamma units (Figure 1).…”

“…Briefly, for each sentence, a 6-channel reduced auditory spectrogram was calculated with a biologically plausible model of the auditory periphery (Chi, Ru and Shamma, 2005). Additionally, slow amplitude modulation of the sentence waveform was calculated following procedures described in Hyafil and colleagues (Hyafil and Cernak, 2015;Hyafil et al, 2015). Syllable boundaries in the input sentences were defined with the Tsylb2 (Fisher, 1996) programme based on the phonemic transcriptions provided in the TIMIT database (Garofolo et al, 1993).…”

“…Neural oscillations, as a proxy of rhythmic collective properties of neurons (Hauk, Giraud and Clarke, 2017;Meyer, Sun and Martin, 2019;Bree et al, 2021), are directly involved in various aspects of speech processing (Zoefel, Archer-Boyd and Davis, 2018;Marchesotti et al, 2020), including speech chunking at different granularity levels depending on their frequency (phrases, words, syllables, phonemic features) and information encoding depending on their cross-frequency interactions (Giraud and Poeppel, 2012;Hyafil et al, 2015;Bonhage et al, 2017;Ghitza, 2020;Proix et al, 2022). Theta (4-7Hz) and low-gamma (25-35Hz) oscillations are related to bottom-up processes, notably the hierarchical encoding of phonemic information within syllables (Hyafil et al, 2015;Mai, Minett and Wang, 2016;Lizarazu, Lallier and Molinaro, 2019). Delta (1-4Hz) and low-beta (14-21Hz) oscillations, which are also frequently observed in relation with speech processing, have a more endogenous origin.…”

Composition of Deep and Spiking Neural Networks for Very Low Bit Rate Speech Coding

“…To estimate syllable boundaries from the speech signal, a neuromorphic oscillatory device is used, based on modelling brain neural oscillations at syllable frequency. This results in highly noise robust incremental syllable boundary detection [54]. It is built around an interconnected network composed of 10 excitatory and 10 inhibitory leaky integrate-and-fire neurons.…”

“…frequencies around formants likely to provide more information about syllable boundaries because of the vocalisation process. Syllable boundaries are characterised by local minima of the weighted signal; these can be generalised to a convolution of the temporal kernel and the weighted signal [54].…”