3D Traffic Scene Understanding From Movable Platforms

Abstract

Abstract-In this paper, we present a novel probabilistic generative model for multi-object traffic scene understanding from movable platforms which reasons jointly about the 3D scene layout as well as the location and orientation of objects in the scene. In particular, the scene topology, geometry and traffic activities are inferred from short video sequences. Inspired by the impressive driving capabilities of humans, our model does not rely on GPS, lidar or map knowledge. Instead, it takes advantage of a diverse set of visual cues in the form of vehicle tracklets, vanishing points, semantic scene labels, scene flow and occupancy grids. For each of these cues we propose likelihood functions that are integrated into a probabilistic generative model. We learn all model parameters from training data using contrastive divergence. Experiments conducted on videos of 113 representative intersections show that our approach successfully infers the correct layout in a variety of very challenging scenarios. To evaluate the importance of each feature cue, experiments using different feature combinations are conducted. Furthermore, we show how by employing context derived from the proposed method we are able to improve over the state-of-the-art in terms of object detection and object orientation estimation in challenging and cluttered urban environments.

Omnidirectional 3D reconstruction in augmented Manhattan worlds

“…We quantitatively show that our model outperforms state-of-the-art stereo matching techniques [12], [10] which have demonstrated superior performance in related evaluations such as the KITTI stereo benchmark [13]. We also show that our results are qualitatively more pleasing as they are less susceptible to noise and allow for identifying the dominant planes which can be useful input information to subsequent higher-level reasoning stages such as scene understanding [14]. Our code, dataset and ground truth depth maps are publicly available 1 .…”

Tracking 3D Moving Objects Based on GPS/IMU Navigation Solution, Laser Scanner Point Cloud and GIS Data

“…A prototype of an autonomous vehicle, known as Google car, has been claimed to be fully operational [18]. However, this approach relies on high accuracy GIS maps, which are not affordable for most companies and institutions [19].…”

Improving Multi-frame Data Association with Sparse Representations for Robust Near-online Multi-object Tracking

“…Among offline techniques, global approaches perform the data association over all the frames simultaneously or by batch [5][6][7][8][9][10][11][12][13][14][15], whereas sliding window (a.k.a. multi-scan, near-online, or online with delay) methods optimize only a few recent frames at the same time [16][17][18][19][20].…”