A Three-Dimensional Deep Learning Framework for Human Behavior Analysis Using Range-Doppler Time Points

Du, Heng; Jin, Tian; Song, Yongping; Dai, Yongpeng; Li, Meng

doi:10.1109/lgrs.2019.2930636

Cited by 24 publications

(23 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(3) MD-SVM [27], a machine learning method that extracts six statistical features from the micro-Doppler spectrogram and feed them into a support vector machine; (4) R-SVM, a support vector machine that takes the SIFT features [58] of the range-time domain as input instead of the micro-Doppler spectrogram; (5) MDR-SA [11], a recently proposed sparse autoencoder that takes both micro-Doppler spectrograms and range profiles as input, and the hidden features of the autoencoder are fed into a an MLP for classification; and (6) MD-SA and (7) R-SA which use the same architecture proposed in [11] and process on the micro-Doppler and range profiles separately. We also compare our model with its naive version, P-Net [15], which consumes the whole range-Doppler-time point sets with PointNet [36]. All these methods are tested on the CMU dataset and the UTD-MHAD datasets.…”

Section: B Classification Performance Comparisonmentioning

confidence: 99%

“…The candidate methods are as follows: (1) PointNet-based methods, namely, P-Net (a three-dimensional deep learning framework that proposed in [15]), P-Net+OSVM (an architecture proposed in [46] that combines PointNet model and a one-class support vector machine), P-Net+Scale (a PointNet model that uses temperature scaling [48]), and P-Net+Adv (a PointNet model that uses temperature scaling and perturbation, which was introduced in Eq. 3); and (2) hierarchical PointNet-based methods, namely, HP-Net (the hierarchical PointNet model introduced in Fig.…”

Section: Overlapping Echo Detectionmentioning

confidence: 99%

“…Recently, there have been some attempts [13], [14] to sense human motions by extracting point clouds in the range-velocity-time space, but the inherent redundancy of these point sets hindered its performance. Du et al [15], [16] developed a 3D deep learning framework for human activity classification. However, their work cannot capture local structures of the micromotion signature in a hierarchical way and has not been conducted in an outlier analysis.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Human Behavior Recognition Using Range-Velocity-Time Points

Chen

2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

Radar-based sensors do not require optimal lighting and atmospheric conditions and nonocclusion, making them a promising solution for human behavior analysis in complex environments. Existing radar-based models generally retrieve features from either the time-velocity domain or the time-range domain. Such two-dimensional representations cannot fully depict dynamic human motion features. In this paper, we propose a temporal range-Doppler PointNet-based method to analyze human behavior. We transform human echoes to 3D point sets and then feed them into the hierarchical PointNet model for classification. The proposed point network can learn structural features from the micromotion trajectory more effectively than directly processing the raw point cloud. To further improve our model's robustness in practical applications, we design an outlier detection module for detecting anomalies such as in multitarget scenarios. The results of experiments on motion capture databases and range-Doppler radar measurements demonstrate that our method realizes outstanding performance in terms of the classification accuracy, noise robustness, and anomaly detection accuracy. INDEX TERMS Human activity recognition, micro-Doppler effect, deep learning, point set, range-Doppler processing, ultra-wideband radar.

show abstract

Section: B Classification Performance Comparisonmentioning

confidence: 99%

Section: Overlapping Echo Detectionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Human Behavior Recognition Using Range-Velocity-Time Points

Chen

2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…Improving neural networks for human activity recognition or deriving detection estimation theories is out of the scope of this study. Figure 1 summarises the state-of-the-art in human activity recognition using radar with either simulated or measured data, considering the different domains of radar data representations based on [1,2,6,24,25]. The research community has mainly focused on mD signatures for human radar classification.…”

Section: Introductionmentioning

confidence: 99%

Simulation framework for activity recognition and benchmarking in different radar geometries

Zhou

Lin

Kernec

et al. 2021

IET Radar Sonar & Navi

View full text Add to dashboard Cite

Radar micro-Doppler signatures have been proposed for human monitoring and activity classification for surveillance and outdoor security, as well as for ambient assisted living in healthcare-related applications. A known issue is the performance reduction when the target is moving tangentially to the line of sight of the radar. Multiple techniques have been proposed to address this, such as multistatic radar and to some extent, interferometric (IF) radar. A simulator is presented to generate synthetic data representative of eight radar systems (monostatic, circular multistatic and in-line multistatic [IM] and IF) to quantify classification performances as a function of aspect angles and deployment geometries. This simulator allows an unbiased performance evaluation of different radar systems. Six human activities are considered with signatures originating from motion-captured data of 14 different subjects. The classification performances are analysed as a function of aspect angles ranging from 0°to 90°per activity and overall. It demonstrates that IF configurations are more robust than IM configurations. However, IM performs better at angles below 55°b efore IF configurations take over. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

show abstract

“…In terms of the technical principle, it solves the detection problems caused by the complex environment such as no light, shielding, and non-line-of-sight [28]. Existing studies on radar sensors mainly use the micro-Doppler effect of radar to classify the movement of human targets [29,30]. However, these studies can only judge the motion state of human targets, and cannot obtain the spatial location information of different body parts of human targets.…”

Section: Introductionmentioning

confidence: 99%

Through-Wall Human Pose Reconstruction via UWB MIMO Radar and 3D CNN

et al. 2021

Self Cite

View full text Add to dashboard Cite

Human pose reconstruction has been a fundamental research in computer vision. However, existing pose reconstruction methods suffer from the problem of wall occlusion that cannot be solved by a traditional optical sensor. This article studies a novel human target pose reconstruction framework using low-frequency ultra-wideband (UWB) multiple-input multiple-output (MIMO) radar and a convolutional neural network (CNN), which is used to detect targets behind the wall. In the proposed framework, first, we use UWB MIMO radar to capture the human body information. Then, target detection and tracking are used to lock the target position, and the back-projection algorithm is adopted to construct three-dimensional (3D) images. Finally, we take the processed 3D image as input to reconstruct the 3D pose of the human target via the designed 3D CNN model. Field detection experiments and comparison results show that the proposed framework can achieve pose reconstruction of human targets behind a wall, which indicates that our research can make up for the shortcomings of optical sensors and significantly expands the application of the UWB MIMO radar system.

show abstract

A Three-Dimensional Deep Learning Framework for Human Behavior Analysis Using Range-Doppler Time Points

Cited by 24 publications

References 24 publications

Human Behavior Recognition Using Range-Velocity-Time Points

Human Behavior Recognition Using Range-Velocity-Time Points

Simulation framework for activity recognition and benchmarking in different radar geometries

Through-Wall Human Pose Reconstruction via UWB MIMO Radar and 3D CNN

Contact Info

Product

Resources

About