Tasking networked CCTV cameras and mobile phones to identify and localize multiple people

Abstract

We present a method to identify and localize people by leveraging existing CCTV camera infrastructure along with inertial sensors (accelerometer and magnetometer) within each person's mobile phones. Since a person's motion path, as observed by the camera, must match the local motion measurements from their phone, we are able to uniquely identify people with the phones' IDs by detecting the statistical dependence between the phone and camera measurements. For this, we express the problem as consisting of a twomeasurement HMM for each person, with one camera measurement and one phone measurement. Then we use a maximum a posteriori formulation to find the most likely ID assignments. Through sensor fusion, our method largely bypasses the motion correspondence problem from computer vision and is able to track people across large spatial or temporal gaps in sensing. We evaluate the system through simulations and experiments in a real camera network testbed.

Gestures Everywhere

“…Sensor fusion has received extensive research across a diverse range of fields in computer science and engineering, including aerospace [1]; robotics [18]; people tracking [17,5]; and pervasive computing [6]. Bayesian-based techniques, such as Kalman filters [4], and particle filters [6,18], are particularly useful as they provide a powerful statistical tool to help manage measurement uncertainty and perform multisensor fusion.…”

A System for Multimodal Context-Awareness

“…Since most of the recent mobile phones contain accelerometers and magnetometers attached to them, mobile phones are considered to be very convenient and fulfill all of the above requirements. In [1] the authors combined an existing CCTV based system with sensors (accelerometers and magnetometers) embedded to a person's mobile phone as a solution. According to this method, the camera captures the location of each person, which is transmitted wirelessly to the mobile phone carried by the respective person.…”

“…The proliferation of ambient intelligent environments has triggered research related to applications, such as monitoring Assistive Daily Living (ADL), fall detection, risk prevention and surveillance [1,2]. For achieving these goals, activity recognition performed in a natural and unintrusive way is of utmost importance.…”

Automatic Synchronization of Wearable Sensors and Video-Cameras for Ground Truth Annotation -- A Practical Approach

“…For example, inertial measurement units (IMU) are used to support computer vision for tracking [1,7], or for localization and identification of objects or persons [8].…”

“…For example, Gu and Tomasi introduced a theoretical framework where the phase disparity between two data streams is explicitly modeled as a so-called Ornstein-Uhlenbeck random process, which is then used to calculate a discriminative measure of synchrony [3]. The computational complexity of the procedure is substantial accelerometers using hidden Markov models [8]. Again the procedure is computationally very demanding and requires rather long trajectories for the analysis.…”