Real-Time Radar-Based Gesture Detection and Recognition Built in an Edge-Computing Platform

Abstract

In this paper, a real-time signal processing framework based on a 60 GHz frequency-modulated continuous wave (FMCW) radar system to recognize gestures is proposed. In order to improve the robustness of the radar-based gesture recognition system, the proposed framework extracts a comprehensive hand profile, including range, Doppler, azimuth and elevation, over multiple measurement-cycles and encodes them into a feature cube. Rather than feeding the range-Doppler spectrum sequence into a deep convolutional neural network (CNN) connected with recurrent neural networks, the proposed framework takes the aforementioned feature cube as input of a shallow CNN for gesture recognition to reduce the computational complexity. In addition, we develop a hand activity detection (HAD) algorithm to automatize the detection of gestures in real-time case. The proposed HAD can capture the time-stamp at which a gesture finishes and feeds the hand profile of all the relevant measurementcycles before this time-stamp into the CNN with low latency. Since the proposed framework is able to detect and classify gestures at limited computational cost, it could be deployed in an edge-computing platform for real-time applications, whose performance is notedly inferior to a state-of-the-art personal computer. The experimental results show that the proposed framework has the capability of classifying 12 gestures in realtime with a high F1-score.

A Low-Complexity Radar Detector Outperforming OS-CFAR for Indoor Drone Obstacle Avoidance

“…The phase shift φ i for target i is given by φ i = π sin θ i [28] where θ i is the angle of arrival of the signal reflected by target i. The IF signals r qm (t) (2) are then sampled by an ADC with a sampling period T f to obtain the discrete-time signals r qm [n] for each chirp q and each antenna m. It is then possible to retrieve the range, Doppler radial velocity and angle of arrival of each target i by computing the Discrete Fourier Transform (DFT) of r qm [n] along n, q and m, respectively (a 3D DFT for each radar frame) [30]. The magnitude of the resulting data cube exhibits N t peaks at the bin locations corresponding to the range, velocity and angle of arrival of each target i in the scene.…”

ASL Trigger Recognition in Mixed Activity/Signing Sequences for RF Sensor-Based User Interfaces

“…While motion detection can be done with a human-in-the-loop approach, this is not desirable in automate, stand-alone systems. Instead, a power-based automated segmentation algorithm, such as short time average over long time average (STA/LTA) [56], [57], dynamic boundary detection (DBD) [58] or power burst curve [59] (PBC) may be utilized.…”

“…Their relative average power is used to define an adaptive threshold value. The STA/LTA method proposed in [57] has proven to be very successful in detecting the tail (end point) of hand gestures. However, the method uses fixed length detection windows, whose duration is selected based on the duration of the longest gesture in the dataset.…”

Synthetic Radar Dataset Generator for Macro-Gesture Recognition

“…In [2], 2750 samples were recorded from 11 subjects, each performing 10 gestures 25 times. Similarly, in [3] the authors collected 7200 samples from 20 subjects which performed 12 gestures 30 times. In both cases, the authors recorded micro-gestures that were performed a few centimeters above the device.…”