Novel Approach for Gesture Recognition Using mmWave FMCW RADAR

Zhao, Yanhua; Sark, Vladica; Krstić, Miloš; Grass, Eckhard

doi:10.1109/vtc2022-spring54318.2022.9860976

Cited by 16 publications

(5 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this study, different tasks are emulated as different datasets, i.e., execution of a single task would mean performing a classification task on a single dataset. Thus, for three tasks, we have considered three datasets (i.e., radar samples for hand gesture detection [46], SVHN [19], MNIST [20]) from two different modalities (i.e., radar and camera image). These three tasks are illustrated in Fig.…”

Section: Workflow For Multiple Tasks Execution On Fused and Branched ...mentioning

confidence: 99%

See 1 more Smart Citation

FPGA Implementation of a Fault-Tolerant Fused and Branched CNN Accelerator With Reconfigurable Capabilities

Syed,

Zhao,

Chen

et al. 2024

IEEE Access

Self Cite

View full text Add to dashboard Cite

The ImageNet moment was a turning point for Convolutional Neural Networks (CNNs), as it demonstrated their potential to revolutionize computer vision tasks. This triumph of CNNs has motivated solving even more complex problems involving multiple tasks from multiple data modalities. Conventionally, a single CNN accelerator has been optimized to perform just one task or multiple correlated tasks. This study presents a shared-layers approach that leverages the pattern-learning capabilities of CNNs to perform multiple uncorrelated tasks from different modalities using a single hardware accelerator. We overcame the challenge of data imbalance in multi-modal learning by synthetic data generation. We achieved an average classification accuracy above 90% on a single CNN accelerator, which would otherwise require three accelerators. Due to the reliability concerns imposed by transistor shrinking and aging, we extended the shared layers methodology and introduced a fault-tolerant CNN accelerator with reconfigurable capabilities supporting fault-tolerant (FT), high-performance (HP), and de-stress (DS) modes. FT mode provides high reliability against soft errors utilizing double/triple modular redundancy, HP mode offers peak performance of 0.979 TOPs using parallel execution, and DS mode reduces dynamic power consumption by up to 68.6% in clock-gated design and even more using a partial reconfiguration method, contributing to decelerating the aging process of the circuit. We have comprehensively evaluated two different CNN architectures (i.e., fused and branched), for three distinct tasks, in three different operating modes, based on accuracy, quantization, pruning, hardware resource utilization, power, energy, performance, and reliability.

show abstract

Section: Workflow For Multiple Tasks Execution On Fused and Branched ...mentioning

confidence: 99%

“…Data preprocessing of T1 consists of collecting the real raw radar data, extracting information by fast Fourier transform (FFT), removing the clutter, and fusing the range, velocity, and angle feature maps. Additional technical details of the real radar dataset are available in [46].…”

Section: A Data Preprocessingmentioning

confidence: 99%

FPGA Implementation of a Fault-Tolerant Fused and Branched CNN Accelerator With Reconfigurable Capabilities

Syed,

Zhao,

Chen

et al. 2024

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…Many researchers have made various contributions to the field of heat map-based hand gesture recognition. Heat map-based gesture recognition is a relatively common approach and in [ 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 ] all authors use heat map-based recognition.…”

Section: Introductionmentioning

confidence: 99%

“…From the raw data of the FMCW radar not only time-distance and time-Doppler heat maps can be extracted, but also a time–angle heat maps. These three types of heat maps are employed as the basis for hand gesture recognition in [ 9 ]. The authors adopted background modelling to separate the static background and the moving target so that the feature heat maps were cleaned efficiently.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Low Complexity Radar Gesture Recognition Using Synthetic Training Data

Zhao

Sark

Krstić

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

Developments in radio detection and ranging (radar) technology have made hand gesture recognition feasible. In heat map-based gesture recognition, feature images have a large size and require complex neural networks to extract information. Machine learning methods typically require large amounts of data and collecting hand gestures with radar is time- and energy-consuming. Therefore, a low computational complexity algorithm for hand gesture recognition based on a frequency-modulated continuous-wave (FMCW) radar and a synthetic hand gesture feature generator are proposed. In the low computational complexity algorithm, two-dimensional Fast Fourier Transform is implemented on the radar raw data to generate a range-Doppler matrix. After that, background modelling is applied to separate the dynamic object and the static background. Then a bin with the highest magnitude in the range-Doppler matrix is selected to locate the target and obtain its range and velocity. The bins at this location along the dimension of the antenna can be utilised to calculate the angle of the target using Fourier beam steering. In the synthetic generator, the Blender software is used to generate different hand gestures and trajectories and then the range, velocity and angle of targets are extracted directly from the trajectory. The experimental results demonstrate that the average recognition accuracy of the model on the test set can reach 89.13% when the synthetic data are used as the training set and the real data are used as the test set. This indicates that the generation of synthetic data can make a meaningful contribution in the pre-training phase.

show abstract