Multi-Feature Encoder for Radar-Based Gesture Recognition

FMCW Radar-Based Hand Gesture Recognition Using Spatiotemporal Deformable and Context-Aware Convolutional 5-D Feature Representation

“…Moreover, although there're already methods considering using the 5D feature representation, such as 3D-CNN (MPCA) [22], 3D-CNN+LSTM [23] and 2D-CNN (multi-feature encoder) [22]. However, our methods outperform all of these methods.…”

“…However, our methods outperform all of these methods. The 3D-CNN (MPCA) only gets an accuracy of 90.22%, and this may be explained by that this method suffers from the high dimensionality of the extracted 5D feature tensor [22]. The 2D-CNN (multi-feature encoder) method gets an accuracy of 94.72% on the test dataset, 4.5% higher than that of 3D-CNN (MPCA), a little (0.4%) higher than that of 3D-CNN+LSTM, comparable with that of S3D (5D feature cubes), but still lower than that of our methods, such as S3D+STDC, S3D+ASTCAC and S3D+STDC+ASTCAC.…”

“…Although there have been methods such as [21]- [23] which explore using the 5D feature representation, their mining of the interrelation among the 5D feature space is not sufficient, and there is still room for improvement. For example, [22][23] use a multi feature encoder to encode the selected the first K points' 5D features of the gesture which have the greatest magnitudes in the range-Doppler spectrograms. However, the fixed manually selected K points are susceptible to dynamic interference and difficult to adapt to complex scenes.…”

“…In terms of feature extraction and classification, traditional methods, such as Hidden Markov Models (HMM) [25], Support Vector Machines (SVM) [26], can only classify a few simple gestures. More complex gesture categories can be recognized via convolutional neural network (CNN) based methods [22]- [24] or Long Short-Term Memory (LSTM) based methods [27] [28] from thousands of data samples. However, using separately extracted range, Doppler and angle information to train CNNs, these methods are usually 2D-CNN based, and they cannot take a 5D feature representation as input.…”

“…However, using separately extracted range, Doppler and angle information to train CNNs, these methods are usually 2D-CNN based, and they cannot take a 5D feature representation as input. Even inputs in [22][23] consider the 5D feature, their used simple CNN networks have difficult in effectively extracting the key information that characterizes different gestures.…”