Structured Inference Networks Using High-Dimensional Sensors for Surveillance Purposes

Polfliet, Vincent; Knudde, Nicolas; Vandersmissen, Baptist; Couckuyt, Ivo; Dhaene, Tom

doi:10.1007/978-3-319-98204-5_2

Cited by 5 publications

(11 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that, with the standard NN-JPDA tracking algorithm, when a track is deleted and re-initialized, it is counted as a mismatch in Eq. (20), significantly lowering the MOTA. Moreover, data association errors can lead to track swaps when the trajectories of two subjects intersect.…”

Section: Tracking Phase Evaluationmentioning

confidence: 96%

“…In the last few years, person identification from backscattered mm-wave radio signals has attracted a considerable and growing interest. Most of the research attention has been paid to processing human micro-Doppler (µD) signatures as a means to distinguish among subjects, usually employing deep learning classifiers, applied to the µD spectrogram [8]- [11], [17]- [20]. Although this approach is robust and accurate, it presents some drawbacks.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Real-Time People Tracking and Identification From Sparse mm-Wave Radar Point-Clouds

Pegoraro

Rossi

2021

IEEE Access

View full text Add to dashboard Cite

Mm-wave radars have recently gathered significant attention as a means to track human movement and identify subjects from their gait characteristics. A widely adopted method to perform the identification is the extraction of the micro-Doppler signature of the targets, which is computationally demanding in case of co-existing multiple targets within the monitored physical space. Such computational complexity is the main problem of state-of-the-art approaches, and makes them inapt for real-time use. In this work, we present an end-to-end, low-complexity but highly accurate method to track and identify multiple subjects in real-time using the sparse point-cloud sequences obtained from a low-cost mm-wave radar. Our proposed system features an extended object tracking Kalman filter, used to estimate the position, shape and extension of the subjects, which is integrated with a novel deep learning classifier, specifically tailored for effective feature extraction and fast inference on radar point-clouds. The proposed method is thoroughly evaluated on an edge-computing platform from NVIDIA (Jetson series), obtaining greatly reduced execution times (reduced complexity) against the best approaches from the literature. Specifically, it achieves accuracies as high as 91.62%, operating at 15 frames per seconds, in identifying three subjects that concurrently and freely move in an unseen indoor environment, among a group of eight.INDEX TERMS mm-wave radar, person identification, point-clouds, multi-target tracking, convolutional neural networks

show abstract

Section: Tracking Phase Evaluationmentioning

confidence: 96%

Section: Related Workmentioning

confidence: 99%

Real-Time People Tracking and Identification From Sparse mm-Wave Radar Point-Clouds

Pegoraro

Rossi

2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…In this study, and setting us apart from the research efforts reviewed above, we compare and combine the use of two different high-dimensional sensors as input for multiple DNNs, with the aim of automatically recognizing a wide range of indoor human activities. To that end, we significantly extend upon the work of [27], in which the subject of automatic activity recognition using a radar and video camera sensor is briefly explored. Specifically, Polfliet et al [27] only partly focuses on activity recognition using a radar and camera sensor by constructing a limited activity data set consisting of 540 samples distributed over three events.…”

Section: Related Workmentioning

confidence: 99%

“…To that end, we significantly extend upon the work of [27], in which the subject of automatic activity recognition using a radar and video camera sensor is briefly explored. Specifically, Polfliet et al [27] only partly focuses on activity recognition using a radar and camera sensor by constructing a limited activity data set consisting of 540 samples distributed over three events. Due to the low number of activities and samples in the data set, only a limited analysis of the effects of combining both sensors is given.…”

Section: Related Workmentioning

confidence: 99%

Indoor human activity recognition using high-dimensional sensors and deep neural networks

Vandersmissen

Knudde

Jalalvand

et al. 2019

Neural Comput & Applic

Self Cite

View full text Add to dashboard Cite

Many smart home applications rely on indoor human activity recognition. This challenge is currently primarily tackled by employing video camera sensors. However, the use of such sensors is characterized by fundamental technical deficiencies in an indoor environment, often also resulting in a breach of privacy. In contrast, a radar sensor resolves most of these flaws and maintains privacy in particular. In this paper, we investigate a novel approach towards automatic indoor human activity recognition, feeding high-dimensional radar and video camera sensor data into several deep neural networks. Furthermore, we explore the efficacy of sensor fusion to provide a solution in less than ideal circumstances. We validate our approach on two newly constructed and published data sets that consist of 2347 and 1505 samples distributed over six different types of gestures and events, respectively. From our analysis, we can conclude that, when considering a radar sensor, it is optimal to make use of a three-dimensional convolutional neural network that takes as input sequential range-Doppler maps. This model achieves 12.22% and 2.97% error rate on the gestures and the events data set, respectively. A pre-trained residual network is employed to deal with the video camera sensor data and obtains 1.67% and 3.00% error rate on the same data sets. We show that there exists a clear benefit in com-Baptist Vandersmissen

show abstract

“…Such performance is achieved in an online fashion (continuous tracking and identification), allowing one to recognize user identities as these share the same physical space, without relying on any visual representation of the scene. We stress that previous work [1], [3], [4] has coped with a single-person identification problem and the multi-user case has only been addressed in an offline fashion through the superposition of multiple single-person signals. In contrast, we build a system that effectively works when multiple persons concurrently share and freely move within the same indoor space, directly working on the composite reflected signal that they generate.…”

Section: Introductionmentioning

confidence: 99%

Multi-Person Continuous Tracking and Identification from mm-Wave micro-Doppler Signatures

Pegoraro,

Meneghello,

Rossi

2020

Preprint

View full text Add to dashboard Cite

In this work, we investigate the use of backscattered mm-wave radio signals for the joint tracking and recognition of identities of humans as they move within indoor environments. Previous research has considered a single-person identification problem, while the multi-person case was only addressed in an offline fashion through the superposition of multiple single-person signals. In contrast, we build a system that effectively works with multiple persons concurrently sharing and freely moving within the same indoor space. This leads to a complicated setting, which requires one to deal with the randomness and complexity of the resulting (composite) backscattered signal. Our solution features a novel signal processing pipeline: first, the signal is filtered to remove artifacts, reflections and random noise that do not originate from humans. A following density-based classification algorithm is executed to separate the Doppler signatures of different users. The last two blocks are trajectory tracking and user identification, respectively based on Kalman filters and deep neural networks. Our results demonstrate that the integration of these two processing stages is critical towards achieving robustness and accuracy in multi-user settings. The proposed system is tested both on a single-target public dataset, for which it outperforms state-of-the-art techniques, and on our own measurements, obtained with a 77 GHz radar on multiple subjects simultaneously moving in an indoor environment. The system works in an online fashion, permitting the continuous identification of multiple subjects with accuracies up to 98%, e.g., with four subjects sharing the same physical space.

show abstract

Structured Inference Networks Using High-Dimensional Sensors for Surveillance Purposes

Cited by 5 publications

References 14 publications

Real-Time People Tracking and Identification From Sparse mm-Wave Radar Point-Clouds

Real-Time People Tracking and Identification From Sparse mm-Wave Radar Point-Clouds

Indoor human activity recognition using high-dimensional sensors and deep neural networks

Multi-Person Continuous Tracking and Identification from mm-Wave micro-Doppler Signatures

Contact Info

Product

Resources

About