Dynamic Hand Gesture Recognition in In-Vehicle Environment Based on FMCW Radar and Transformer

Zheng, Lianqing; Bai, Jie; Zhu, Xichan; Huang, Libo; Shan, Chewu; Wu, Qiong; Zhang, Lei

doi:10.3390/s21196368

Cited by 22 publications

(21 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Figure 4 a shows this method. One other novel result for recognizing single gesture motion is from [ 13 ], where a Transformer-encoder architecture is used by learning the temporal features from time series of the Range-Doppler Map (RDM), Range Azimuth Map (RAM), and Range Elevation Map (REM), and these temporal features are fused as a fused feature. Figure 4 b displays the method.…”

Section: Preliminariesmentioning

confidence: 99%

“…In the following, we describe the positional embedding operation, based on our 2D/3D embedded features. Let

be the position embedding function [ 13 ] for the 2D embedded features

and

where

and

are position embedding after performing the

function to the 2D embedded features

, and

, respectively.…”

Section: Our Proposed Chmr-hs Algorithmmentioning

confidence: 99%

“…For HMR and continuous HMR, the Doppler signature and the Range-Doppler-Time signature from FMCW radar are the most common extracted features in the existing literature [ 10 , 11 , 12 , 13 ]. Papanastasiou et al.…”

Section: Introductionmentioning

confidence: 99%

“…For instance, Zheng et al. [ 13 ] proposed a dynamic hand gesture recognition algorithm by extracting the features of the Range-Doppler Map (RDM), Range-Azimuth Map (RAM), and Range-Elevation Map (REM) of time signatures as an input of the trained model.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Multi-Feature Transformer-Based Learning for Continuous Human Motion Recognition with High Similarity Using mmWave FMCW Radar

Chen

Cheng

et al. 2022

Sensors

View full text Add to dashboard Cite

Doppler-radar-based continuous human motion recognition recently has attracted extensive attention, which is a favorable choice for privacy and personal security. Existing results of continuous human motion recognition (CHMR) using mmWave FMCW Radar are not considered the continuous human motion with the high similarity problem. In this paper, we proposed a new CHMR algorithm with the consideration of the high similarity (HS) problem, called as CHMR-HS, by using the modified Transformer-based learning model. As far as we know, this is the first result in the literature to investigate the continuous HMR with the high similarity. To obtain the clear FMCW radar images, the background and target signals of the detected human are separated through the background denoising and the target extraction algorithms. To investigate the effects of the spectral-temporal multi-features with different dimensions, Doppler, range, and angle signatures are extracted as the 2D features and range-Doppler-time and range-angle-time signatures are extracted as the 3D features. The 2D/3D features are trained into the adjusted Transformer-encoder model to distinguish the difference of the high-similarity human motions. The conventional Transformer-decoder model is also re-designed to be Transformer-sequential-decoder model such that Transformer-sequential-decoder model can successfully recognize the continuous human motions with the high similarity. The experimental results show that the accuracy of our proposed CHMR-HS scheme are 95.2% and 94.5% if using 3D and 2D features, the simulation results also illustrates that our CHMR-HS scheme has advantages over existing CHMR schemes.

show abstract

Section: Preliminariesmentioning

confidence: 99%

“…In the following, we describe the positional embedding operation, based on our 2D/3D embedded features. Let

be the position embedding function [ 13 ] for the 2D embedded features

and

where

and

are position embedding after performing the

function to the 2D embedded features

, and

, respectively.…”

Section: Our Proposed Chmr-hs Algorithmmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multi-Feature Transformer-Based Learning for Continuous Human Motion Recognition with High Similarity Using mmWave FMCW Radar

Chen

Cheng

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…A CNN feature extractor with an attention-based network was used in [19] for person and activity recognition. In [20], a CNN and a vision-transformer were used for HGR for in-vehicle environments. In these transformer networks, the output feature map of the CNN was directly fed as the input to the transformer.…”

Section: Introductionmentioning

confidence: 99%

Vision Transformer with Convolutional Encoder–Decoder for Hand Gesture Recognition using 24-GHz Doppler Radar

et al. 2022

View full text Add to dashboard Cite

Transformers combined with convolutional encoders have been recently used for hand gesture recognition (HGR) using micro-Doppler signatures. We propose a vision-transformer-based architecture for HGR with multi-antenna continuous-wave Doppler radar receivers. The proposed architecture consists of three modules: a convolutional encoderdecoder, an attention module with three transformer layers, and a multi-layer perceptron. The novel convolutional decoder helps to feed patches with larger sizes to the attention module for improved feature extraction. Experimental results obtained with a dataset corresponding to a two-antenna continuous-wave Doppler radar receiver operating at 24 GHz (published by Skaria et al.) confirm that the proposed architecture achieves an accuracy of 98.3% which substantially surpasses the state-of-the-art on the used dataset.

show abstract

Radar Sensing in Healthcare: Challenges and Achievements in Human Activity Classification & Vital Signs Monitoring

Fioranelli

Guendel

Kruse

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Dynamic Hand Gesture Recognition in In-Vehicle Environment Based on FMCW Radar and Transformer

Cited by 22 publications

References 36 publications

Multi-Feature Transformer-Based Learning for Continuous Human Motion Recognition with High Similarity Using mmWave FMCW Radar

Multi-Feature Transformer-Based Learning for Continuous Human Motion Recognition with High Similarity Using mmWave FMCW Radar

Vision Transformer with Convolutional Encoder–Decoder for Hand Gesture Recognition using 24-GHz Doppler Radar

Radar Sensing in Healthcare: Challenges and Achievements in Human Activity Classification & Vital Signs Monitoring

Contact Info

Product

Resources

About