Recovery and Reasoning About Occlusions in 3D Using Few Cameras with Applications to 3D Tracking

Abstract

In this work we propose algorithms to learn the locations of static occlusions and reason about both static and dynamic occlusion scenarios in multi-camera scenes for 3D surveillance (e.g., reconstruction, tracking). We will show that this leads to a computer system which is able to more effectively track (follow) objects in video when they are obstructed from some of the views. Because of the nature of the application area, our algorithm will be under the constraints of using few cameras (no more than 3) that are configured wide-baseline. Our algorithm consists of a learning phase, where a 3D probabilistic model of occlusions is estimated per-voxel, per-view over time via an iterative framework. In this framework, at each frame the visual hull of each foreground object (person) is computed via a Markov Random Field that integrates the occlusion model. The model is then updated at each frame using this solution, providing an iterative process that can accurately estimate the occlusion model over time and overcome the few-camera constraint. We demonstrate the application of such a model to a number of areas, including visual hull reconstruction, the reconstruction of the occluding structures themselves, and 3D tracking.

Occlusion-aware 3D multiple object tracker with two cameras for visual surveillance

“…However, in order not to confuse the tracked object when silhouettes of other objects get close by, each tracker keeps an appearance of tracked object per-view. When objects are occluded dynamically in a camera view, this appearance gets corrupted [13] and centroid of the merged silhouette deviates, which cause good particle hypotheses diminish. Occlusion filter is designed to keep track of dynamic occlusions in each view separately so that appearance update is stopped and increased measurement error is taken into account during occlusions.…”

“…We neither put a constraint on camera placement nor assume that objects are moving on a ground plane. Synchronization of cameras need not be perfect as opposed to [8,9,13,14].…”

“…Fusing tracking results [4] is prone to errors from inconsistent or redundant tracking results. Fusing features from all views into a common view is another approach, examples of which sample the scene space into grids [8], voxels [9,13] or stack of parallel planes [14] and evaluate the occupancy of those samples. These visual hullbased algorithms are prone to phantom targets when 2 cameras are utilized [9,13,14].…”

“…Keck et al [13] proposed a per-voxel per-view occlusion model for handling of static occlusions. They ignored dynamic occlusions reasoning that visual hull method reconstructs dynamically occluded objects.…”

Recovery of Temporal Synchronization Error through Online 3D Tracking with Two Cameras

“…Many algorithms have been proposed [5,9,6,8], however, they all have been tested under limited intervals of time due to the synchronization requirement. In addition, camera synchronization becomes problematic when the distances between cameras and speeds of objects increase (e.g.…”