Human Activity Classification Based on Point Clouds Measured by Millimeter Wave MIMO Radar With Deep Recurrent Neural Networks

Kim, Youngwook; Alnujaim, Ibrahim; Oh, Daegun

doi:10.1109/jsen.2021.3068388

Cited by 56 publications

(42 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Specifically, the usage of multiple channels in an antenna array allows estimating the angles of arrival of the targets. At mm-wave frequencies, human bodies are perceived as extended targets, with multiple scatterers generated by each moving body part forming the so-called point clouds (PCs) [23]. This scattering behavior, combined with the angular estimation capabilities on both azimuth and elevation, enables a new, broader 'feature space' to explore for radar-based HAR.…”

Section: Description Of the Proposed Pipeline And Comparative Baselinesmentioning

confidence: 99%

“…Each coherent processing interval (frame duration) is 100ms. PCs from 20 frames are aggregated to represent one segment of activity, since the PC generated from a single coherent processing interval is typically not dense enough to represent the shape of a human body [23]. (b) spectrograms are generated by applying a Short Time Fourier Transform (STFT) on the slow-time axis aggregated over the range bins where the human subject is present.…”

Section: Description Of the Proposed Pipeline And Comparative Baselinesmentioning

confidence: 99%

“…While effective, this approach requires the usage of multiple nodes, with an increase in complexity of the overall system and the need to cope with the synchronization of data from the different sensor nodes. Another approach, recently proposed thanks to the availability of millimeter-wave (mm-wave) MIMO (Multiple Input Multiple Output) radars, is based on the use of 4D imaging radar that exploits azimuth and elevation resolution capabilities to attain additional spatial information on the subject's posture [23], [24]. While promising, this approach is still not investigated in detail in the literature and there is a scope to define effective classification processing pipelines that can take the advantages of mm-wave 4D imaging capabilities for HAR.…”

Section: Introductionmentioning

confidence: 99%

“…• Unlike in the other studies such as [23] where the radar point clouds are treated as images, the proposed pipeline is designed with the goal to exploit all the six intrinsic 'features' obtained by imaging radar, namely range, azimuth, elevation, Doppler, received power, and time information, rather than focusing on just one specific data representation. • The hierarchical structure of the proposed pipeline simplifies the task of HAR in a multi-angle scenario, with the help of the designated neural networks, T-Net, and achieves classification of both static postures and dynamic motions.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Angle-Insensitive Human Motion and Posture Recognition Based on 4D Imaging Radar and Deep Learning Classifiers

2022

View full text Add to dashboard Cite

The need for technologies for Human Activity Recognition (HAR) in home environments is becoming more and more urgent because of the aging population worldwide. Radar-based HAR is typically using micro-Doppler signatures as one of the main data representations, in conjunction with classification algorithms often inspired from deep learning methods. One of the limitations of this approach is the challenging classification of movements at unfavorable aspect angles (i.e., close to 90 • ) and of static postures in between continuous sequences of activities. To address this problem, a hierarchical processing and classification pipeline is proposed to fully exploit all the information available from millimeter-wave (mm-wave) 4D imaging radars, specifically the azimuth and elevation information in conjunction to the more conventional range, Doppler, received power, and time features. The proposed pipeline uses the two complementary data representations of Point Cloud (PC) and spectrogram, and its performance is validated using an experimental dataset with 6 activities performed by 8 participants. The results show good performance of the proposed pipeline compared with alternative baseline approaches in the literature, and the effect of key parameters such as the amount of training data, signal-to-noise levels, and virtual aperture size is investigated. Leave-one-subject-out test is also applied to study the impact of body characteristics on the generalizability of the trained classifiers.

show abstract

Section: Description Of the Proposed Pipeline And Comparative Baselinesmentioning

confidence: 99%

Section: Description Of the Proposed Pipeline And Comparative Baselinesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Angle-Insensitive Human Motion and Posture Recognition Based on 4D Imaging Radar and Deep Learning Classifiers

2022

View full text Add to dashboard Cite

show abstract

“…In FMCW radar, the global spatial distribution and general intensity of people with different activities was first extracted by the averaging energy feature from RD i . In addition, considering that various activities in the 3-D space lead to dissimilar dynamic scattering in both the transverse and longitudinal angle dimensions, a 2-D angle-FFT [25] was conducted to obtain the azimuth-elevation angle spectrum for feature extraction.…”

Section: Energy and Lbp Feature Extraction On Fmcw Radarmentioning

confidence: 99%

Indoor Activity and Vital Sign Monitoring for Moving People with Multiple Radar Data Fusion

et al. 2021

View full text Add to dashboard Cite

The monitoring of human activity and vital signs plays a significant role in remote health-care. Radar provides a non-contact monitoring approach without privacy and illumination concerns. However, multiple people in a narrow indoor environment bring dense multipaths for activity monitoring, and the received vital sign signals are heavily distorted with body movements. This paper proposes a framework based on Frequency Modulated Continuous Wave (FMCW) and Impulse Radio Ultra-Wideband (IR-UWB) radars to address these challenges, designing intelligent spatial-temporal information fusion for activity and vital sign monitoring. First, a local binary pattern (LBP) and energy features are extracted from FMCW radar, combined with the wavelet packet transform (WPT) features on IR-UWB radar for activity monitoring. Then the additional information guided fusing network (A-FuseNet) is proposed with a modified generative and adversarial structure for vital sign monitoring. A Cascaded Convolutional Neural Network (CCNN) module and a Long Short Term Memory (LSTM) module are designed as the fusion sub-network for vital sign information extraction and multisensory data fusion, while a discrimination sub-network is constructed to optimize the fused heartbeat signal. In addition, the activity and movement characteristics are introduced as additional information to guide the fusion and optimization. A multi-radar dataset with an FMCW and two IR-UWB radars in a cotton tent, a small room and a wide lobby is constructed, and the accuracies of activity and vital sign monitoring achieve 99.9% and 92.3% respectively. Experimental results demonstrate the superiority and robustness of the proposed framework.

show abstract

Radar Sensing in Healthcare: Challenges and Achievements in Human Activity Classification & Vital Signs Monitoring

Fioranelli

Guendel

Kruse

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Human Activity Classification Based on Point Clouds Measured by Millimeter Wave MIMO Radar With Deep Recurrent Neural Networks

Cited by 56 publications

References 30 publications

Angle-Insensitive Human Motion and Posture Recognition Based on 4D Imaging Radar and Deep Learning Classifiers

Angle-Insensitive Human Motion and Posture Recognition Based on 4D Imaging Radar and Deep Learning Classifiers

Indoor Activity and Vital Sign Monitoring for Moving People with Multiple Radar Data Fusion

Radar Sensing in Healthcare: Challenges and Achievements in Human Activity Classification & Vital Signs Monitoring

Contact Info

Product

Resources

About