Slanted Stixels: A Way to Represent Steep Streets

Abstract

This work presents and evaluates a novel compact scene representation based on Stixels that infers geometric and semantic information. Our approach overcomes the previous rather restrictive geometric assumptions for Stixels by introducing a novel depth model to account for non-flat roads and slanted objects. Both semantic and depth cues are used jointly to infer the scene representation in a sound global energy minimization formulation.Furthermore, a novel approximation scheme is introduced in order to significantly reduce the computational complexity of the Stixel algorithm, and then achieve real-time computation capabilities. The idea is to first perform an over-segmentation of the image, discarding the unlikely Stixel cuts, and apply the algorithm only on the remaining Stixel cuts. This work presents a novel over-segmentation strategy based on a Fully Convolutional Network (FCN), which outperforms an approach based on using local extrema of the disparity map.We evaluate the proposed methods in terms of semantic and geometric accuracy as well as run-time on four publicly available benchmark datasets. Our approach maintains accuracy on flat road scene datasets while improving substantially on a novel non-flat road dataset.

3D Perception With Slanted Stixels on GPU

“…Our work builds upon the proposal from [3]: they use semantic cues in addition to depth to extract a Stixel representation, which is able to provide a rich yet compact representation of the traffic scene. We also base our method on [12]: the Slanted Stixels model incorporates a novel plane model together with effective priors on the plane parameters, and it is able to represent scenes with complex non-flat roads.…”

“…The Stixel world has been successfully used for representing traffic scenes, as introduced in [2]. The field of intelligent vehicles has been using this model over the last years [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13]. The Stixel world defines a compact representation of the dense 3D disparity data obtained from stereo vision that uses rectangles, the so called Stixels, as elements.…”

“…The Slanted Stixel world model recently proposed in [11], [12] generalizes the original proposal with a more flexible plane model that overcomes the previous rather restrictive constant depth and constant height assumptions for object and ground Stixels, respectively, and accurately represents arbitrary kinds of slanted objects and non-flat roads. Basically, the Slanted model defines a plane in the disparity space defined by two random variables: line slope and intercept.…”

“…2048-by-1024-pixel images scaled 4 and 8 times are segmented, respectively, at 0.7 and 6.6 frames per second (fps) on a six-core Intel i7-6800K processor [12]. These performance results do not meet the real-time or energy efficiency requirements of autonomous driving applications.…”

“…This chart illustrates the trade-off between accuracy and run-time. We compare Slanted Stixels [12] and our method, both with Stixel resolution 4×4 and 8×8. Disparity error corresponds to results for SYNTHIA-SF [11], and runtime was measured on the NVIDIA Tegra Xavier embedded GPU.…”