Through-Wall Human Mesh Recovery Using Radio Signals

Zhao, M.; Liu, Yingcheng; Raghu, Aniruddh; Zhao, Hang; Li, Tianhong; Torralba, Antonio; Katabi, Dina

doi:10.1109/iccv.2019.01021

Cited by 87 publications

(53 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table 5 summarizes the power consumption of some systems. SDR-based systems have achieved through-wall localization [1], [5], [46], through-wall imaging [9], [10], [47], [48], and human vital signals detection [2], [49]. The advantage of such systems is that signal processing methods are relatively flexible but the disadvantage is the high cost of special devices like USRP.…”

Section: Software-defined Radio (Sdr) Systemmentioning

confidence: 99%

“…Tadar, proposed by Yang et al, leverages an RFID reader and multiple RFID tags to locate people behind a wall [46]. RF-Pose [10], RF-Pose3D [47], and RF-Avatar [48], proposed by Zhao et al, can reconstruct the 2D or 3D model of a human body through RF signals. These three systems combine RF devices and camera devices, and leverage machine learning techniques to train RF systems supervised by vision systems.…”

Section: Software-defined Radio (Sdr) Systemmentioning

confidence: 99%

“…However, feeding the whole snapshot to a fully connected neural network is intractable since such a dense neural network would contain a huge number of parameters which are hard to train and easy to overfit. Therefore, CNN is used to reduce the complexity and extract only important features from the input RF signals and then uses these features to predict the positions of targets [10], [47], [48].…”

Section: Machine Learningmentioning

confidence: 99%

“…Applying 4-dimensional convolution operations to extract features from RF signals directly is time-consuming and even mainstream open-source machine learning libraries (e.g., Tensorflow) only support up to 3-dimensional convolution operations. Therefore, Zhao et al proposed a tensor decomposition technique to reduce the convolution dimension and leveraged auxiliary neural networks to help CNN focus on the regions with targets to reduce the computational complexity [47], [48].…”

Section: Machine Learningmentioning

confidence: 99%

See 3 more Smart Citations

A Survey of Handy See-Through Wall Technology

Luan

Zhu

et al. 2020

IEEE Access

View full text Add to dashboard Cite

The through-wall system, which applies radio technologies to detect objects behind the wall, can find many appealing applications such as public security, life detection, and medical health monitoring. While being studied for years, the recent advances in high-performance handheld computing devices and artificial intelligence have made the through-wall system more practical. In this article, we present a tutorial-like study on the fundamental radio technologies used in the through-wall system, as well as its recent advances. Different from the traditional through-wall radars, this paper mainly focuses on the handy through-wall techniques with low power, narrow bandwidth, lightweight, no contact, and civil use. Advanced through-wall systems and open research issues are also presented. INDEX TERMS Seeing through walls, machine learning, motion capture, wireless.

show abstract

Section: Software-defined Radio (Sdr) Systemmentioning

confidence: 99%

Section: Software-defined Radio (Sdr) Systemmentioning

confidence: 99%

Section: Machine Learningmentioning

confidence: 99%

Section: Machine Learningmentioning

confidence: 99%

See 2 more Smart Citations

A Survey of Handy See-Through Wall Technology

Luan

Zhu

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…We use a type of radio commonly used in past work on RF-based action recognition [45,24,41,7,28,33,16,42,46,44]. The radio generates a waveform called FMCW and operates between 5.4 and 7.2 GHz.…”

Section: Radio Frequency Signals Primermentioning

confidence: 99%

Making the Invisible Visible: Action Recognition Through Walls and Occlusions

Li¹,

Fan²,

Zhao³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

111

View full text Add to dashboard Cite

Understanding people's actions and interactions typically depends on seeing them. Automating the process of action recognition from visual data has been the topic of much research in the computer vision community. But what if it is too dark, or if the person is occluded or behind a wall? In this paper, we introduce a neural network model that can detect human actions through walls and occlusions, and in poor lighting conditions. Our model takes radio frequency (RF) signals as input, generates 3D human skeletons as an intermediate representation, and recognizes actions and interactions of multiple people over time. By translating the input to an intermediate skeleton-based representation, our model can learn from both vision-based and RF-based datasets, and allow the two tasks to help each other. We show that our model achieves comparable accuracy to vision-based action recognition systems in visible scenarios, yet continues to work accurately when people are not visible, hence addressing scenarios that are beyond the limit of today's vision-based action recognition.

show abstract

In-Home Daily-Life Captioning Using Radio Signals

Fan¹,

Li²,

Yuan³

et al. 2020

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

This paper aims to caption daily life -i.e., to create a textual description of people's activities and interactions with objects in their homes. Addressing this problem requires novel methods beyond traditional video captioning, as most people would have privacy concerns about deploying cameras throughout their homes. We introduce RF-Diary, a new model for captioning daily life by analyzing the privacy-preserving radio signal in the home with the home's floormap. RF-Diary can further observe and caption people's life through walls and occlusions and in dark settings. In designing RF-Diary, we exploit the ability of radio signals to capture people's 3D dynamics, and use the floormap to help the model learn people's interactions with objects. We also use a multi-modal feature alignment training scheme that leverages existing video-based captioning datasets to improve the performance of our radio-based captioning model. Extensive experimental results demonstrate that RF-Diary generates accurate captions under visible conditions. It also sustains its good performance in dark or occluded settings, where video-based captioning approaches fail to generate meaningful captions. 1

show abstract

Through-Wall Human Mesh Recovery Using Radio Signals

Cited by 87 publications

References 37 publications

A Survey of Handy See-Through Wall Technology

A Survey of Handy See-Through Wall Technology

Making the Invisible Visible: Action Recognition Through Walls and Occlusions

In-Home Daily-Life Captioning Using Radio Signals

Contact Info

Product

Resources

About