RGB-W: When Vision Meets Wireless

Abstract

Inspired by the recent success of RGB-D cameras, we propose the enrichment of RGB data with an additional "quasi-free" modality, namely, the wireless signal emitted by individuals' cell phones, referred to as RGB-W. The received signal strength acts as a rough proxy for depth and a reliable cue on a person's identity. Although the measured signals are noisy, we demonstrate that the combination of visual and wireless data significantly improves the localization accuracy. We introduce a novel image-driven representation of wireless data which embeds all received signals onto a single image. We then evaluate the ability of this additional data to (i) locate persons within a sparsitydriven framework and to (ii) track individuals with a new confidence measure on the data association problem. Our solution outperforms existing localization methods. It can be applied to the millions of currently installed RGB cameras to better analyze human behavior and offer the next generation of high-accuracy location-based services.

MonoLoco: Monocular 3D Pedestrian Localization and Uncertainty Estimation

“…Kundegorski and Breckon [29] achieved reasonable performances combining infrared imagery and realtime photogrammetry. Alahi et al combined monocular images with wireless signals [3] or with additional visual priors [1,2]. The seminal work of Mono3D [8] exploited deep learning to create 3D object proposals for car, pedestrian and cyclist categories but it did not evaluate 3D localization of pedestrians.…”

Nowhere to Hide: Cross-modal Identity Leakage between Biometrics and Devices

“…Data association is widely used in radar systems, when tracking blips on a radar screen [10], as well as in object monitoring of surveillance systems [24]. When it comes to the cross-modal association, research attention is limited and all dedicated to location tracking of humans [6,49,59]. These methods heavily rely on the hypothesis that both sensor modalities are observing evolving state spaces matched precisely in the temporal domain.…”

How Effectively can Indoor Wireless Positioning Relieve Visual Tracking Pains: A Cramer-Rao Bound Viewpoi

“…Complete dataset can be found on our website. 1 In order to study how the goodness of wireless positioning will impact the visual tracking performance, we consider a BLE network with N = 32 beacons and simulate Receive Signal Strength (RSS) measurements according to the model given in Section 3 subject to Gaussian noise with zero mean and different noise variances σ 2 v . We use the Mean-Square-Error (MSE) as a measure of wireless positioning accuracy.…”

“…As pointed out in [1], wireless data and visual data are complementary and should be fused for enhanced tracking performance. In the ideal cases where the above mentioned hurdles do not occur, visual tracking is in general more accurate and informative than wireless positioning.…”