Refining Weakly-Supervised Free Space Estimation Through Data Augmentation and Recursive Training

Abstract

Free space estimation is an important problem for autonomous robot navigation. Traditional camera-based approaches rely on pixel-wise ground truth annotations to train a segmentation model. To cover the wide variety of environments and lighting conditions encountered on roads, training supervised models requires large datasets. This makes the annotation cost prohibitively high. In this work, we propose a novel approach for obtaining free space estimates from images taken with a single road-facing camera. We rely on a technique that generates weak free space labels without any supervision, which are then used as ground truth to train a segmentation model for free space estimation. We study the impact of different data augmentation techniques on the performances of free space predictions, and propose to use a recursive training strategy. Our results are benchmarked using the Cityscapes dataset and improve over comparable published work across all evaluation metrics. Our best model reaches 83.64% IoU (+2.3%), 91.75% Precision (+2.4%) and 91.29% Recall (+0.4%). These results correspond to 88.8% of the IoU, 94.3% of the Precision and 93.1% of the Recall obtained by an equivalent fully-supervised baseline, while using no ground truth annotation. Our code and models are freely available online.

Striving for Less: Minimally-Supervised Pseudo-Label Generation for Monocular Road Segmentation

“…Rather than explicitly relying on scene geometry, another successful method is to identify the road by over-segmenting frames into superpixels and extracting feature vectors for each superpixel using a network trained for generic image classification. Superpixels can then be clustered in feature-space before using a spatial prior to identify the cluster corresponding to the road [4,6,7]. Our unsupervised approach unifies the geometrical and semantic approaches by combining features extracted from a v-disparity representation with cues obtained from over-segmenting the RGB space into superpixels.…”

“…In order to obtain our final pseudo-labels, we also remove Unsupervised model training: We use the pseudo-labels as targets to train a model to segment the road directly from an RGB input. Recent work has shown that deep neural networks trained via stochastic gradient descent exhibit surprising robustness to noise in their training targets [20], and such procedure has repeatedly been proven beneficial to road segmentation [4]- [7]. The network architecture and training procedure are detailed in Section V.…”

“…Our experiments leverage the Cityscapes dataset, which provides pixel-wise ground truth labels for 30 visual classes in 5000 frames [1]. Since the official test set has no public annotation, we follow prior work [5]- [7] and use the 500 frames of the validation set as our test set. Pseudo-labels Generator Pseudo-Labels concat + Fig.…”

“…We use the Cutmix data augmentation strategy proposed for road segmentation in [7] for all of our models. For models that use RGB as inputs, we also apply the Color-Crop-Flip (CFC) augmentation strategy, which applies a color jitter, takes a random crop of appropriate aspect ratio, and randomly perform an horizontal flip of the input [7]. Each augmentation is applied with 50% probability.…”

“…Although unsupervised approaches lead to respectable results, they are not reliable enough to enable the safe operation of autonomous agents [3]- [7]. Semi-supervised methods offer a compromise by training on restricted subsets of manually-labeled data and exploiting unlabeled data through self-supervised consistencies or pseudo-labeling [8]- [10].…”