TS-I3D Based Hand Gesture Recognition Method With Radar Sensor

Abstract

Aiming at the problems of the noise impact on the parametric image of hand gestures, the difficulty of gesture feature extraction, and the inefficient utilization of continuous gesture time sequential information, we propose a time sequential inflated 3 dimensions (TS-I3D) convolutional neural network approach for hand gesture recognition based on frequency modulated continuous wave (FMCW) radar sensor. Specifically, the FMCW radar is used to acquire the hand gesture data, and the range and speed of the gesture in each frame signal are calculated by 2 dimensions fast Fourier transform. Then, the range-Doppler map (RDM) is generated based on the relationship between motion parameters and frequency. The interference in RDM caused by people and the external environment is filtered out and the peak of hand gesture in RDM is further enhanced by wavelet transform. Finally, TS-I3D network is designed to extract the range and speed change information of the continuous gestures. The experimental results show that the average recognition accuracy rate of the hand gestures of the proposed method is 96.17%. INDEX TERMS FMCW radar, hand gesture recognition, interference filtering, deep learning, LSTM. I. INTRODUCTION With the development of advanced signal processing approaches and deep learning [1], [2], human-computer interaction has attracted much attention. Hand gesture recognition (HGR), as one of the most important way of human-computer interaction, gets rid of the limitations of people in traditional control devices, and has been widely used in various fields, i.e., smart home like switching TV channels, turning on the air conditioner or intelligent driving like answering a phone call. The existing HGR methods mainly contain three stages, namely hand gesture data acquisition, feature extraction, and gesture classification. Hand gesture data is mainly collected using optics camera [3]-[5] or radar [6]-[9]. Optical-based gesture recognition mainly applies cameras to capture gesture images, and then applies machine-learning methods [10], [11] for feature extraction and recognition. Coelho et al. [4] use Kinect to capture RGB and depth images of hand gestures, The associate editor coordinating the review of this manuscript and approving it for publication was Guan Gui.

Hand Gesture Recognition Using an IR-UWB Radar with an Inception Module-Based Classifier

“…These types of UWB signaling-based radars are typically referred to as impulse-radio ultra-wideband (IR-UWB) radars [10]. Other types of typical radars used for hand motion sensing and classification are frequency modulated continuous wave (FMCW) radars [11,12] and Doppler radars [13].…”

“…Previously, researchers paid little attention to evaluating these inception modules for radar signal classification. Wang and co-workers [12] implemented the GoogLeNet classifier to classify gestures using an FMCW radar. In the case of UWB radars, researchers have not treated the acquired gesture data in three-dimensional (3D) way.…”

Real-Time Radar-Based Gesture Detection and Recognition Built in an Edge-Computing Platform

“…Thanks to the wide available bandwidth (7 GHz), their systems could recognize fine hand motions. Similarly, the authors in [15]- [17] also extracted hand motions based on RD spectrums via an FMCW radar. In [18], [19], apart from the range and Doppler information of hand gestures, they also considered the incident angle information by using multiple receive antennas to enhance the classification accuracy of their gesture recognition system.…”

“…For example, the authors in [12], [18]- [20] considered the TDF spectrums or range profiles as images and directly fed them into a deep convolutional neural network (CNN). Whereas, other research works [14], [15], [21] considered the radar data over multiple measurement-cycles 1558-1748 ©2020 IEEE. Personal use of this material is permitted.…”

A Novel Detection and Recognition Method for Continuous Hand Gesture Using FMCW Radar

“…However, with the number of hand gestures increases, the instability of the trigger ratio makes it difficult to detect hand gestures correctly. For the hand gesture recognition algorithms, deep learning [24][25][26][27][28] is usually used for feature extraction and hand gesture classification. Since deep learning method requires a large number of datasets for training, and the hand gesture collection time and the training time increase sharply with the increase of hand gesture types.…”