Classification and Semantic Mapping of Urban Environments

Abstract

In this paper we address the problem of classifying objects in urban environments based on laser and vision data. We propose a framework based on Conditional Random Fields (CRFs), a flexible modeling tool allowing spatial and temporal correlations between laser returns to be represented. Visual features extracted from color imagery as well as shape features extracted from 2D laser scans are integrated in the estimation process. The paper contains the following novel developments: (1) a probabilistic formulation for the problem of exploiting spatial and temporal dependencies to improve classification; (2) three methods for classification in 2D semantic maps; (3) a novel semi-supervised learning algorithm to train CRFs from partially labeled data; (4) the combination of local classifiers with CRFs to perform feature selection on high-dimensional feature vectors. The system is extensively evaluated on two different datasets acquired in two different cities with different sensors. An accuracy of 91% is achieved on a seven-class problem. The classifier is also applied to the generation of a 3 km long semantic map.

Detection-based object labeling in 3D scenes

“…Thus far, most robotic scene understanding work has focused on outdoor scenes captured with laser scanners, where technologies have matured and enabled high-impact applications such as mapping and autonomous driving [2], [25], [21], [7], [30]. Indoor scenes, in comparison, prove more challenging and cover a wider range of objects, scene layouts, and scales.…”

“…One way to label a multiframe scene would be to directly work with the merged point cloud. Point cloud labeling has worked very well for outdoor scenes [21], [7], [30], and the recent work of [1] illustrates how labeling can be done on indoor 3D scenes, where local features are computed from segments of point clouds and integrated in a Markov Random Field (MRF) (a) (b) Fig. 1.…”

Recursive Inference for Prediction of Objects in Urban Environments

“…In these methods the graph structure is typically induced by a partitioning of 3D point clouds. Authors in [5] consider 2D semantic mapping of street scenes with laser and image data providing computationally expensive solution with a graph induced by Delaunay triangulation. Both laser and image measurements are used in [20], where efficient solution is provided considering only vehicle object class.…”

Co-inference for Multi-modal Scene Analysis

“…The conventional way to approach this problem is to constrain the representation into only one of the modalities while integrating information from the other discarded domain as features. That is, the approach can be 2-D driven [5,[8][9][10][11][12]1], in that reasoning is done in the image while integrating 3-D features, or the approach can be 3-D driven [7,[13][14][15], in that the predictions are made on the 3-D data while integrating 2-D features. These approaches are typically only applicable when the two modalities are in correspondence.…”