Deep Spiking Neural Networks for Large Vocabulary Automatic Speech Recognition

Abstract

Artificial neural networks (ANN) have become the mainstream acoustic modeling technique for large vocabulary automatic speech recognition (ASR). A conventional ANN features a multi-layer architecture that requires massive amounts of computation. The brain-inspired spiking neural networks (SNN) closely mimic the biological neural networks and can operate on low-power neuromorphic hardware with spike-based computation. Motivated by their unprecedented energyefficiency and rapid information processing capability, we explore the use of SNNs for speech recognition. In this work, we use SNNs for acoustic modeling and evaluate their performance on several large vocabulary recognition scenarios. The experimental results demonstrate competitive ASR accuracies to their ANN counterparts, while require significantly reduced computational cost and inference time. Integrating the algorithmic power of deep SNNs with energy-efficient neuromorphic hardware, therefore, offer an attractive solution for ASR applications running locally on mobile and embedded devices.

Allosteric Regulation at the Crossroads of New Technologies: Multiscale Modeling, Networks, and Machine Learning

“…The remarkable rise of deep learning (DL) relying on the robust function approximations and representation properties of deep neural networks has provided us with new tools to automatically find compact low-dimensional representations (features) of high-dimensional data (LeCun et al, 2015 ). DL models have achieved outstanding predictive performance making dramatic breakthroughs in a wide range of applications, including automatic speech processing and image recognition (Toledano et al, 2018 ; Kim et al, 2019 ; Hey et al, 2020 ; Wu et al, 2020 ). In the words of Geoffrey Hinton who is the founder of DL technologies “Deep Learning is an algorithm which has no theoretical limitations on what it can learn; the more data you give and the more computational time you provide the better it is” (LeCun et al, 2015 ).…”

Acoustic Modeling in Speech Recognition: A Systematic Review

“…A review paper was presented to address acoustic modeling issues and refinements [37]. The first constructs and functioning of HMM and its constraints reviewed.…”

Low-Activity Supervised Convolutional Spiking Neural Networks Applied to Speech Commands Recognition

“…There is a growing interest for SNNs applied to speech recognition tasks, from isolated word and phone recognition [3,4,5,6],to large-vocabulary automatic speech recognition (ASR) very recently [7]. Reasons are that the audio speech signal is particularly suited to event-driven models such as SNNs, SNNs are also more biologically realistic than DNNs, hardware friendly and energy efficient models, if implemented on dedicated energy-efficient neuromorphic chips.…”

“…We explore the Leaky Integrate-and-Fire (LIF) neuron model for this task, and show that convolutional SNNs can reach an accuracy very close to the one obtained with state-ofthe-art DNNs, for this task. Our main contributions are the following: i) we propose to use dilated convolution spiking layers, ii) we define a new regularization term to penalize the averaged number of spikes to keep the spiking neuron activity as sparse as possible, iii) we show that the leaky variant of the neuron model outperforms the non-leaky one (NLIF), used in [7].…”