Synthetic Training Data Generator for Hand Gesture Recognition Based on FMCW RADAR

Zhao, Yanhua; Sark, Vladica; Krstić, Miloš; Grass, Eckhard

doi:10.23919/irs54158.2022.9904997

Cited by 4 publications

(7 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The features of our sample are saved in a csv [ 32 ] file. In Table 5 , it can be observed that the features extracted from a sample of [ 20 ] are 8 to 12 times larger than the features extracted by our new approach. Compared to [ 17 , 18 , 19 , 20 ], the average recognition accuracy on the real data set is higher, which indicates the strength of this work.…”

Section: Experiments and Evaluationmentioning

confidence: 99%

“…In Table 5 , it can be observed that the features extracted from a sample of [ 20 ] are 8 to 12 times larger than the features extracted by our new approach. Compared to [ 17 , 18 , 19 , 20 ], the average recognition accuracy on the real data set is higher, which indicates the strength of this work. In addition to this, our synthetic feature generator does not contain radar signal simulation, which reduces the computational effort significantly.…”

Section: Experiments and Evaluationmentioning

confidence: 99%

“…Reference [ 20 ] simulated range–time, velocity–time and angle–time heatmap features for six types of hand gestures by combining gestures constructed in Blender and a simulated FMCW radar. For both, the classifiers were trained with synthetic data sets and the models were tested with real data sets.…”

Section: Experiments and Evaluationmentioning

confidence: 99%

“…In view of this, the gesture synthetic training data generator proposed by the authors in [ 20 ] can generate range–time heat maps, velocity–time heat maps and angle–time heat maps. Six gestures were synthesised and real data were also employed to test their validity.…”

Section: Introductionmentioning

confidence: 99%

“…A simplified synthetic hand gesture feature generator is presented. As the hand gesture features extracted from the real data are simplified to one dimension, the synthesis processes from [ 15 , 17 , 18 , 19 , 20 ] are no longer needed. In our simplified synthetic hand gesture feature generator, the generation of radar raw data is skipped.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Low Complexity Radar Gesture Recognition Using Synthetic Training Data

Zhao

Sark

Krstić

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

Developments in radio detection and ranging (radar) technology have made hand gesture recognition feasible. In heat map-based gesture recognition, feature images have a large size and require complex neural networks to extract information. Machine learning methods typically require large amounts of data and collecting hand gestures with radar is time- and energy-consuming. Therefore, a low computational complexity algorithm for hand gesture recognition based on a frequency-modulated continuous-wave (FMCW) radar and a synthetic hand gesture feature generator are proposed. In the low computational complexity algorithm, two-dimensional Fast Fourier Transform is implemented on the radar raw data to generate a range-Doppler matrix. After that, background modelling is applied to separate the dynamic object and the static background. Then a bin with the highest magnitude in the range-Doppler matrix is selected to locate the target and obtain its range and velocity. The bins at this location along the dimension of the antenna can be utilised to calculate the angle of the target using Fourier beam steering. In the synthetic generator, the Blender software is used to generate different hand gestures and trajectories and then the range, velocity and angle of targets are extracted directly from the trajectory. The experimental results demonstrate that the average recognition accuracy of the model on the test set can reach 89.13% when the synthetic data are used as the training set and the real data are used as the test set. This indicates that the generation of synthetic data can make a meaningful contribution in the pre-training phase.

show abstract

Section: Experiments and Evaluationmentioning

confidence: 99%

Section: Experiments and Evaluationmentioning

confidence: 99%

Section: Experiments and Evaluationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Low Complexity Radar Gesture Recognition Using Synthetic Training Data

Zhao

Sark

Krstić

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

FPGA Implementation of a Fault-Tolerant Fused and Branched CNN Accelerator With Reconfigurable Capabilities

Syed,

Zhao,

Chen

et al. 2024

IEEE Access

Self Cite

View full text Add to dashboard Cite

The ImageNet moment was a turning point for Convolutional Neural Networks (CNNs), as it demonstrated their potential to revolutionize computer vision tasks. This triumph of CNNs has motivated solving even more complex problems involving multiple tasks from multiple data modalities. Conventionally, a single CNN accelerator has been optimized to perform just one task or multiple correlated tasks. This study presents a shared-layers approach that leverages the pattern-learning capabilities of CNNs to perform multiple uncorrelated tasks from different modalities using a single hardware accelerator. We overcame the challenge of data imbalance in multi-modal learning by synthetic data generation. We achieved an average classification accuracy above 90% on a single CNN accelerator, which would otherwise require three accelerators. Due to the reliability concerns imposed by transistor shrinking and aging, we extended the shared layers methodology and introduced a fault-tolerant CNN accelerator with reconfigurable capabilities supporting fault-tolerant (FT), high-performance (HP), and de-stress (DS) modes. FT mode provides high reliability against soft errors utilizing double/triple modular redundancy, HP mode offers peak performance of 0.979 TOPs using parallel execution, and DS mode reduces dynamic power consumption by up to 68.6% in clock-gated design and even more using a partial reconfiguration method, contributing to decelerating the aging process of the circuit. We have comprehensively evaluated two different CNN architectures (i.e., fused and branched), for three distinct tasks, in three different operating modes, based on accuracy, quantization, pruning, hardware resource utilization, power, energy, performance, and reliability.

show abstract