Free-Space Detection with Self-Supervised and Online Trained Fully Convolutional Networks

Abstract

Recently, vision-based Advanced Driver Assist Systems have gained broad interest. In this work, we investigate free-space detection, for which we propose to employ a Fully Convolutional Network (FCN). We show that this FCN can be trained in a selfsupervised manner and achieve similar results compared to training on manually annotated data, thereby reducing the need for large manually annotated training sets. To this end, our selfsupervised training relies on a stereo-vision disparity system, to automatically generate (weak) training labels for the color-based FCN. Additionally, our self-supervised training facilitates online training of the FCN instead of offline. Consequently, given that the applied FCN is relatively small, the free-space analysis becomes highly adaptive to any traffic scene that the vehicle encounters. We have validated our algorithm using publicly available data and on a new challenging benchmark dataset that is released with this paper. Experiments show that the online training boosts performance with 5% when compared to offline training, both for F max and AP.

Enhanced free space detection in multiple lanes based on single CNN with scene identification

“…In literature, many approaches exploit visual and LIDAR data jointly with geometric properties of the scene to filter obstacles and detect interesting areas [11]. Others [12], [13] use a CNN, training it on labeled images where the road is annotated. In a recent work, Caltagirone et al [14] developed a data fusion strategy to use either image and LIDAR data in a deep learning based algorithm, and obtain state-of-the-art performances on the KITTI dataset benchmarks for road detection.…”

Detecting unexpected obstacles for self-driving cars: Fusing deep learning and geometric modeling

“…Following a very similar idea, [17] recently proposed a method for free-space detection in urban environments with a self-supervised and online trained Fully Convolutional Network (FCN) [18]. In this case, the automatic ground truth collection process is based on a combination of disparity maps and Stixel-based ground masks computed from a calibrated stereo-pair.…”

Minimizing Supervision for Free-Space Segmentation

“…While an early study [17] trains a probabilistic model, other papers train FCNs [28,35,37,43]. Saleh et al [35] develop a video segmentation algorithm for general background objects including free-space on a road.…”

“…Laddha et al [28] use external maps of the road indexed against the vehicle position according to GPS. Sanberg et al [37] and Guo et al [17] use stereo information for automatically generating segmentation masks. We distinguish our work from these studies in that we only use a collection of monocular vehicle-centric images, which makes our approach even less supervised than most others.…”