Joint Probabilistic People Detection in Overlapping Depth Images

Abstract

Privacy-preserving high-quality people detection is a vital computer vision task for various indoor scenarios, e.g. people counting, customer behavior analysis, ambient assisted living or smart homes. In this work a novel approach for people detection in multiple overlapping depth images is proposed. We present a probabilistic framework utilizing a generative scene model to jointly exploit the multi-view image evidence, allowing us to detect people from arbitrary viewpoints. Our approach makes use of meanfield variational inference to not only estimate the maximum a posteriori (MAP) state but to also approximate the posterior probability distribution of people present in the scene. Evaluation shows state-of-the-art results on a novel data set for indoor people detection and tracking in depth images from the top-view with high perspective distortions. Furthermore it can be demonstrated that our approach (compared to the the monoview setup) successfully exploits the multi-view image evidence and robustly converges in only a few iterations. INDEX TERMS Depth sensor indoor surveillance, depth sensor networks, generative scene model, joint multi-view person detection, mean-field variational inference, multi-camera person detection, people detection in top-view, vertical top-view pedestrian detection.

Crowd Density Estimation by Using Attention Based Capsule Network and Multi-Column CNN

“…PrimaryCaps are reshaped and vectorized. Thus, using the ReLU and tanh activation functions, it is simulated to a fully connected neural network and calculated as in equations ( 11) and (12). Here 1 and 2 are the results of two fully connected layers, 3 and 4 are the weight matrices of these layers, and 3 and 4 offset values.…”

Temporal Smoothing for Joint Probabilistic People Detection in a Depth Sensor Network

“…In previous work [13] we re-cast the problem of people detection and tracking with multiple depth sensors as an inverse problem, employing an approximately differentiable scene model to detect people from arbitrary viewpoints. Following these ideas we introduced a probabilistic framework [1] based on a discrete scene configuration space. For stochastic inference a variational mean-field approximiation is used to jointly exploit the multi-view information in order to estimate the marginal probability distribution of people present in the scene.…”

“…This has three major implications: (i) the sensors capture the scene from the top-view to reduce occlusions in crowded scenes; (ii) since the resulting field of view of a single sensor is quite limited, sensor networks need to be employed to cover a larger area; (iii) position changes of people lead to drastically varying appearances as a result of the vertical top-view. In previous work [1] we propose a probabilistic framework which uses a generative scene model to leverage the full image evidence from all sensor views. For the final approximation of the probability distribution of people present in the scene a mean-field variational inference optimization is employed.…”

“…In this work we investigate how the temporal context can be used to improve the detection performance by regularizing the underlying stochastic optimization process. Hence we present a novel extension of [1] which incorporates the temporal context given by a time series of multi-view depth observations (see Fig. 1).…”

Asymmetric Adaptive Fusion in a Two-Stream Network for RGB-D Human Detection

“…As the second step, the head-shoulder descriptor (HSD) was constructed to jointly encode depth difference information and local geometric information. FUJIMOTO et al [ 18 ] used the roundness and size of a height-continuous region to describe the human upper-back shape. They utilized implicitly included information in the missing region to alleviate the influences of partial loss of depth information caused by occlusions.…”