Making the Invisible Visible: Action Recognition Through Walls and Occlusions

Li, Tianhong; Fan, Linlin; Zhao, M.; Liu, Yingcheng; Katabi, Dina

doi:10.1109/iccv.2019.00096

Cited by 106 publications

(59 citation statements)

References 42 publications

(72 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We use an FMCW radio widely used in previous work on RF-based human sensing [56,27,51,9,34,40,17,53,57,55,24]. The radio is equipped with two antenna arrays: horizontal and vertical.…”

Section: Radio Frequency Signals Primermentioning

confidence: 99%

“…These features are relatively stable over days and months, enabling a more robust longterm ReID system. Moreover, previous works have shown the possibility of tracking people's 3D skeletons and walking patterns using RF signals, which can be used as longterm identifying features [55,24,56,54,33]. Furthermore, unlike RGB-based person ReID methods which struggle in the presence of occlusion and poor lighting, RF signals traverse walls and can enable ReID through occlusions and in dark settings.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning Longterm Representations for Person Re-Identification Using Radio Signals

Fan¹,

Li²,

Fang³

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

View full text Add to dashboard Cite

Person Re-Identification (ReID) aims to recognize a person-of-interest across different places and times. Existing ReID methods rely on images or videos collected using RGB cameras. They extract appearance features like clothes, shoes, hair, etc. Such features, however, can change drastically from one day to the next, leading to inability to identify people over extended time periods. In this paper, we introduce RF-ReID, a novel approach that harnesses radio frequency (RF) signals for longterm person ReID. RF signals traverse clothes and reflect off the human body; thus they can be used to extract more persistent human-identifying features like body size and shape. We evaluate the performance of RF-ReID on longitudinal datasets that span days and weeks, where the person may wear different clothes across days. Our experiments demonstrate that RF-ReID outperforms state-of-the-art RGB-based ReID approaches for long term person ReID. Our results also reveal two interesting features: First since RF signals work in the presence of occlusions and poor lighting, RF-ReID allows for person ReID in such scenarios. Second, unlike photos and videos which reveal personal and private information, RF signals are more privacy-preserving, and hence can help extend person ReID to privacy-concerned domains, like healthcare.

show abstract

“…We use an FMCW radio widely used in previous work on RF-based human sensing [56,27,51,9,34,40,17,53,57,55,24]. The radio is equipped with two antenna arrays: horizontal and vertical.…”

Section: Radio Frequency Signals Primermentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Learning Longterm Representations for Person Re-Identification Using Radio Signals

Fan¹,

Li²,

Fang³

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

View full text Add to dashboard Cite

show abstract

“…In fact, it offers better tolerance to partial occlusion by capturing local aspect using a histogram group of gradients from neighbour regions. Li et al [33] tackled detecting human actions through walls and occlusions using Wi-Fi signals together with deep learning techniques. Relying on radio signals, they could precisely distinguish actions and interactions despite limited lighting conditions using solely radio frequency (RF) signals as input.…”

Section: Occlusionmentioning

confidence: 99%

A Novel Spatio-Temporal Violence Classification Framework Based on Material Derivative and LSTM Neural Network

Lejmi¹,

Khalifa²,

Mahjoub³

2020

View full text Add to dashboard Cite

In the current era, the implementation of automated security video surveillance systems is particularly needy in terms of human violence recognition. Nevertheless, the latter encounters various interlinked difficulties which require efficient solutions as well as feasible methods that provide a relevant distinction between normal human actions and abnormal ones. In this paper, we present an overview of these issues and a literature review of the related works and current research on-going efforts on this field and suggests a novel prediction model for violence recognition, based on a preliminary spatio-temporal features extraction using the material derivative which describes the rate of change of a particle while in motion with respect to time. The classification algorithm is then carried out using a deep learning LSTM technique to classify generated features into eight specified violent and non-violent categories and a prediction value for each class of action is calculated. The whole model is trained on a public dataset and its classification capacity is evaluated on a confusion matrix which assembles all the predictions made by the system with their actual labels. The obtained results are promising and show that the proposed model can be potentially useful for detecting human violence.

show abstract

“…Wireless sensing to capture shape. Radar systems can use RF reflections to detect and track humans [5,37,29]. However, they typically only track location and movements and cannot generate accurate or dynamic body meshes.…”

Section: Related Workmentioning

confidence: 99%

Through-Wall Human Mesh Recovery Using Radio Signals

Zhao¹,

Liu²,

Raghu³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

View full text Add to dashboard Cite

Person behind the wallFigure 1: Dynamic human meshes estimated using radio signals. Images captured by a camera co-located with the radio sensor are presented here for visual reference. (a) shows the estimated human meshes of the same person in sportswear, a baggy costume and when he is behind the wall. (b) shows the dynamic meshes that capture the motion when the person walks, waves his hand, and sits.

show abstract

Making the Invisible Visible: Action Recognition Through Walls and Occlusions

Cited by 106 publications

References 42 publications

Learning Longterm Representations for Person Re-Identification Using Radio Signals

Learning Longterm Representations for Person Re-Identification Using Radio Signals

A Novel Spatio-Temporal Violence Classification Framework Based on Material Derivative and LSTM Neural Network

Through-Wall Human Mesh Recovery Using Radio Signals

Contact Info

Product

Resources

About