Fusion of range measurements and semantic estimates in an evidential framework / Fusion von Distanzmessungen und semantischen Größen im Rahmen der Evidenztheorie

Abstract

A detailed reconstruction of the environment is a crucial component of mobile robotic systems and enables higher level scene understanding. To achieve information redundancy heterogenous sensors need to be used with each sensor having specific strengths and weaknesses. Therefore, the goal of this work is to fuse information from multiple lidars, radars, a stereo camera and semantic camera information into one common scene representation. In contrast to past publications, we focus on the combination of distance measurements and semantic estimates in the image domain in one common evidential framework. Grid maps are used as common fusion structure which enable efficient data processing. The approach is validated on an automated driving plattform in real traffic scenarios. Experiments show that the scene reconstruction precision increases while still retaining the real-time capability.

Semantic Evidential Grid Mapping Using Monocular and Stereo Cameras

“…This makes it particularly suitable for the fusion of semantic estimates with range measurements in top-view as outlined in [ 1 ]. For the BBA, we consider the hypotheses set consisting of singletons as all hypotheses combinations are either conflicting by definition or not estimated by the semantic labeling.…”

“…As introduced in [ 1 ], we calculate the BBA based on the label histogram, which resembles the contribution of accumulated pixels to a class in a given grid cell. We use a u-disparity grid and a u-depth grid to compute the label histogram for stereo vision-based and monocular vision-based grid mapping, respectively.…”

“…Furthermore, cameras are superior when it comes to understanding semantic details in the environment. In [ 1 ], we presented a semantic evidential fusion approach for multi-layer grid maps by introducing a refined set of hypotheses that allows the joint modeling of occupancy and semantic states in a common representation. In this work, we use the same evidence theoretical framework and present two improved sensor models for stereo vision and monocular vision that can be incorporated in the sensor data fusion presented in [ 1 ].…”

“…In contrast to many past publications on vision-based grid mapping like [ 6 , 7 , 8 , 10 , 12 ], we use a wide range of different semantic classes, which can be provided by vision. Instead of assigning only one semantic label per grid cell as in [ 11 , 14 , 15 ], we use the hypotheses set introduced in [ 1 ] to model uncertainty for eight semantic hypotheses in a consistent evidential framework. In order to achieve a dense and smooth BBA for ground hypotheses, we make use of encapsulated ground surface estimations to approximate the pixel-to-area correspondence in the top-view space.…”

“…In order to achieve a dense and smooth BBA for ground hypotheses, we make use of encapsulated ground surface estimations to approximate the pixel-to-area correspondence in the top-view space. The resulting semantic evidential multi-layer grid map can then be fused with range sensor-based grid maps, as described in [ 1 ].…”