Radar-Based Hand Gesture Recognition Using Spiking Neural Networks

Tsang, Ing Jyh; Corradi, Federico; Sifalakis, Manolis; Leekwijck, Werner Van; Latré, Steven

doi:10.3390/electronics10121405

Cited by 29 publications

(18 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This shows that the proposed extensions are general improvements upon the theoretical base LSM implementation (Maass et al, 2002 ). Unlike our previous research on LSMs (Tsang et al, 2021 ), where the optimal input-liquid connection probability is derived separately for every data set, in this work an empirically set rule of thumb is used. The input-liquid connection probability is fixed in such a way that every liquid neuron is connected to ~7 randomly picked input neurons.…”

Section: Resultsmentioning

confidence: 99%

“…Considering the low training complexity and speed, as well as their compatibility with deployment on efficient neuromorphic hardware (Li et al, 2020 ; Wang et al, 2022 ), LSMs have become an attractive SNN model for low-power edge computing. Furthermore, despite their simple setup, LSMs have been established as a powerful spatio-temporal feature extractor with remarkable results on various tasks (Al Zoubi et al, 2018 ; Soures and Kudithipudi, 2019 ) even surpassing the performance of a large CNN+LSTM model on a radar-based gesture recognition benchmark (Tsang et al, 2021 ).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Extended liquid state machines for speech recognition

Deckers

Tsang²,

Leekwijck³

et al. 2022

Front. Neurosci.

Self Cite

View full text Add to dashboard Cite

A liquid state machine (LSM) is a biologically plausible model of a cortical microcircuit. It exists of a random, sparse reservoir of recurrently connected spiking neurons with fixed synapses and a trainable readout layer. The LSM exhibits low training complexity and enables backpropagation-free learning in a powerful, yet simple computing paradigm. In this work, the liquid state machine is enhanced by a set of bio-inspired extensions to create the extended liquid state machine (ELSM), which is evaluated on a set of speech data sets. Firstly, we ensure excitatory/inhibitory (E/I) balance to enable the LSM to operate in edge-of-chaos regime. Secondly, spike-frequency adaptation (SFA) is introduced in the LSM to improve the memory capabilities. Lastly, neuronal heterogeneity, by means of a differentiation in time constants, is introduced to extract a richer dynamical LSM response. By including E/I balance, SFA, and neuronal heterogeneity, we show that the ELSM consistently improves upon the LSM while retaining the benefits of the straightforward LSM structure and training procedure. The proposed extensions led up to an 5.2% increase in accuracy while decreasing the number of spikes in the ELSM up to 20.2% on benchmark speech data sets. On some benchmarks, the ELSM can even attain similar performances as the current state-of-the-art in spiking neural networks. Furthermore, we illustrate that the ELSM input-liquid and recurrent synaptic weights can be reduced to 4-bit resolution without any significant loss in classification performance. We thus show that the ELSM is a powerful, biologically plausible and hardware-friendly spiking neural network model that can attain near state-of-the-art accuracy on speech recognition benchmarks for spiking neural networks.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Extended liquid state machines for speech recognition

Deckers

Tsang²,

Leekwijck³

et al. 2022

Front. Neurosci.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Both approaches turn the range-Doppler maps to spikes by thresholding. Instead, Tsang et al (2021) feeds the spiketrains into a liquid state machine, a recurrent network of spiking neurons retaining a memory of received input, and evaluates various classifiers as read-out: Using an SVM a state-ofthe-art accuracy for SoLi of greater than 98% is reached, which is superior to any DNN approach. For a non-public radar gesture dataset, in Kreutz et al (2021) we combine the AoA information with range-Doppler maps from multiple frames to train deep SNNs with surrogate gradients and temporal coding.…”

Section: Target Classificationmentioning

confidence: 99%

Automotive Radar Processing With Spiking Neural Networks: Concepts and Challenges

et al. 2022

View full text Add to dashboard Cite

Frequency-modulated continuous wave radar sensors play an essential role for assisted and autonomous driving as they are robust under all weather and light conditions. However, the rising number of transmitters and receivers for obtaining a higher angular resolution increases the cost for digital signal processing. One promising approach for energy-efficient signal processing is the usage of brain-inspired spiking neural networks (SNNs) implemented on neuromorphic hardware. In this article we perform a step-by-step analysis of automotive radar processing and argue how spiking neural networks could replace or complement the conventional processing. We provide SNN examples for two processing steps and evaluate their accuracy and computational efficiency. For radar target detection, an SNN with temporal coding is competitive to the conventional approach at a low compute overhead. Instead, our SNN for target classification achieves an accuracy close to a reference artificial neural network while requiring 200 times less operations. Finally, we discuss the specific requirements and challenges for SNN-based radar processing on neuromorphic hardware. This study proves the general applicability of SNNs for automotive radar processing and sustains the prospect of energy-efficient realizations in automated vehicles.

show abstract

“…Along with the continuous improvement of neuromorphic chips, SNN-based solutions have emerged in recent years for various applications and sensors, ranging from speech recognition with resonate-and-fire neurons [9], object tracking for monocular vision [10,11], object detection using raw temporal pulses of lidar sensors [12] for lane keeping Time-coded spiking FT [13], feature extraction and motion perception [14], and collision avoidance based on data obtained from a dynamic vision sensor [15]. Currently, the most prominent task addressed in radar data processing using SNNs is gesture recognition [16,17,18,19]. Micro-Doppler signatures of hand movement are particularly suited for gestures and contain temporal information, which on the other hand leverages the recurrence ability of SNNs.…”

Section: Introductionmentioning

confidence: 99%

Time-coded Spiking Fourier Transform in Neuromorphic Hardware

López-Randulfe,

Reeb,

Karimi

et al. 2022

Preprint

View full text Add to dashboard Cite

After several decades of continuously optimizing computing systems, the Moore's law is reaching its end. However, there is an increasing demand for fast and efficient processing systems that can handle large streams of data while decreasing system footprints. Neuromorphic computing answers this need by creating decentralized architectures that communicate with binary events over time. Despite its rapid growth in the last few years, novel algorithms are needed that can leverage the potential of this emerging computing paradigm and can stimulate the design of advanced neuromorphic chips. In this work, we propose a time-based spiking neural network that is mathematically equivalent to the Fourier transform. We implemented the network in the neuromorphic chip Loihi and conducted experiments on five different real scenarios with an automotive frequency modulated continuous wave radar. Experimental results validate the algorithm, and we hope they prompt the design of ad hoc neuromorphic chips that can improve the efficiency of state-of-the-art digital signal processors and encourage research on neuromorphic computing for signal processing.

show abstract

Radar-Based Hand Gesture Recognition Using Spiking Neural Networks

Cited by 29 publications

References 39 publications

Extended liquid state machines for speech recognition

Extended liquid state machines for speech recognition

Automotive Radar Processing With Spiking Neural Networks: Concepts and Challenges

Time-coded Spiking Fourier Transform in Neuromorphic Hardware

Contact Info

Product

Resources

About