Improving the Accuracy of Spiking Neural Networks for Radar Gesture Recognition Through Preprocessing

Abstract

Event based neural networks are currently being explored as efficient solutions for performing AI task at the extreme edge. To fully exploit their potential, event based neural networks coupled to adequate data pre-processing must be investigated. Within this context, we demonstrate a 4-bit-weight Spiking Neural Network (SNN) for radar gesture recognition is presented, achieving state-of-the-art 93% accuracy within only 4 processing time-steps while using only one convolutional layer and two fully-connected layers. This consumes very little energy and area if implemented in event-based hardware, which makes it suited for embedded extreme-edge applications. In addition, we demonstrate the importance of data pre-processing for achieving a high recognition accuracy in SNNs compared to Deep Neural Networks (DNNs) with the same network topology and training strategy. We show that efficient pre-processing prior to the neural network is drastically more important for SNNs compared to DNNs. We demonstrate for the first time that pre-processing parameters can affect SNNs and DNNs in antagonistic ways, prohibiting the generalization of conclusions drawn from DNN design to SNNs. We demonstrate our findings by rigorously comparing gesture recognition accuracy achieved with our SNN to a DNN with the same architecture and similar training. Unlike previously proposed neural networks for radar processing, this work enables ultra-low-power radar-based gesture recognition for extreme-edge devices.

Fail-Safe Human Detection for Drones Using a Multi-Modal Curriculum Learning Approach

“…at its transmit antenna (where q is the chirp index, f c is the start frequency and α is the slope), and senses the reflected waves through a receive antenna array, providing not only ranging but also angle-of-arrival (AoA) information [20]. The received signal at each receive antenna is demodulated and the following signal is obtained [21]:…”

“…where m is the receive antenna index, N t is the number of reflecting targets, φ i is the phase shift of target i due to its AoA, and T di is the round-trip time from the target i to the radar antenna, linked to the distance d i between the radar and the target following T di = 2di c (with c the speed of light). Through successive FFT processing steps, a range-Doppler-azimuth (RDA) heatmap can be obtained for each radar frame (a packet of N s chirps) [21]. Peaks in the RDA heatmap indicate targets at certain ranges and AoAs, and with a certain radial (or Doppler) velocities.…”

Automotive Radar Processing With Spiking Neural Networks: Concepts and Challenges

“…Other SNNs operate on the micro-Doppler patterns: For the IMEC 8GHz dataset, Stuijt et al (2021) treat the micro-Doppler as a binary image, train a DNN and convert it to a rate-based SNN. For the same dataset, Safa et al (2021b) improve the classification accuracy by means of time-to-first-spike coding, a direct training of the spiking CNN and further preprocessing. Instead, in Arsalan et al (2021), we treat the micro-Doppler pattern as a sequence of velocity vectors which is then fed into a SNN consisting of a 1D convolution layer, one dense LIF hidden layer, and an output layer.…”

“…In both cases, the 2D convolutional layers extract spatial information from single frames while the recurrent layer combines the latter for spatio-temporal signal processing. The proposed SNN model resembles the spiking convolutional network for gesture classification from Safa et al (2021b); yet it was developed independently and differs from it by having recurrent connections between the LIF neurons. Both the ANN and SNN in this work are trained on real-valued inputs and on event-encoded inputs.…”

Time-coded Spiking Fourier Transform in Neuromorphic Hardware

“…Along with the continuous improvement of neuromorphic chips, SNN-based solutions have emerged in recent years for various applications and sensors, ranging from speech recognition with resonate-and-fire neurons [9], object tracking for monocular vision [10,11], object detection using raw temporal pulses of lidar sensors [12] for lane keeping Time-coded spiking FT [13], feature extraction and motion perception [14], and collision avoidance based on data obtained from a dynamic vision sensor [15]. Currently, the most prominent task addressed in radar data processing using SNNs is gesture recognition [16,17,18,19]. Micro-Doppler signatures of hand movement are particularly suited for gestures and contain temporal information, which on the other hand leverages the recurrence ability of SNNs.…”