A Comparison of Low-Complexity Real-Time Feature Extraction for Neuromorphic Speech Recognition

Abstract

This paper presents a real-time, low-complexity neuromorphic speech recognition system using a spiking silicon cochlea, a feature extraction module and a population encoding method based Neural Engineering Framework (NEF)/Extreme Learning Machine (ELM) classifier IC. Several feature extraction methods with varying memory and computational complexity are presented along with their corresponding classification accuracies. On the N-TIDIGITS18 dataset, we show that a fixed bin size based feature extraction method that votes across both time and spike count features can achieve an accuracy of 95% in software similar to previously report methods that use fixed number of bins per sample while using ~3× less energy and ~25× less memory for feature extraction (~1.5× less overall). Hardware measurements for the same topology show a slightly reduced accuracy of 94% that can be attributed to the extra correlations in hardware random weights. The hardware accuracy can be increased by further increasing the number of hidden nodes in ELM at the cost of memory and energy.

How the Brunswikian Lens Model Illustrates the Relationship Between Physiological and Behavioral Signals and Psychological Emotional and Cognitive States

“…In addition, an integrated tool for processing speech, voice, and in real-time would be beneficial. Although realtime speech (Gao et al, 2019), voice (Acharya et al, 2018), and video processing (Ananthanarayanan et al, 2017) and their integration (Kose and Saraclar, 2021) have been widely studied in computer science, the analysis of physiological and behavioral signals in psychological, emotional, and cognitive states and relational messages presents a new and interesting path, especially for real-time applications (e.g., decision support in business negotiations). Another useful future direction is to harness the power of computer-based techniques to perform real-time audio and video quality checks for better data inputs in a non-laboratory setting.…”

Event-Driven Local Gain Control on a Spiking Cochlea Sensor

“…Different spike features have been been proposed for driving the subsequent classifiers. These features include constant time spike count, interspike interval histograms and constant count features [18], [19], [27].…”

“…the Dynamic Audio Sensor [2]) and software cochlea models are being tested as front-ends for audio tasks such as source localization [13], [14], speech recognition [15], speaker verification, multimodal recognition [16] and keyword spotting [17]. Spike features such as constant time bin and constant spike count features [18], [19] are usually presented as inputs to a machine learning classifier such as the SVM classifier. A few studies show that using such features instead of conventional features such as log-filter banks or Mel Frequency Cepstral Coefficient (MFCC) features can lead to similar accuracies for a speech recognition task.…”

Is my Neural Network Neuromorphic? Taxonomy, Recent Trends and Future Directions in Neuromorphic Engineering

“…While being great pieces of research, we feel that this is fundamentally not a good application for SNN since the original input signal is static and does not change with time. Instead, it might be more natural to use SNN as dynamical systems to track moving objects in video streams [18], [19] or classify signals that vary over time such as speech [20]. With this in mind, we propose the following desired characteristics for neuromorphic benchmarks:…”