Addressing appearance change in outdoor robotics with adversarial domain adaptation

Abstract

Appearance changes due to weather and seasonal conditions represent a strong impediment to the robust implementation of machine learning systems in outdoor robotics. While supervised learning optimises a model for the training domain, it will deliver degraded performance in application domains that underlie distributional shifts caused by these changes. Traditionally, this problem has been addressed via the collection of labelled data in multiple domains or by imposing priors on the type of shift between both domains. We frame the problem in the context of unsupervised domain adaptation and develop a framework for applying adversarial techniques to adapt popular, state-of-the-art network architectures with the additional objective to align features across domains. Moreover, as adversarial training is notoriously unstable, we first perform an extensive ablation study, adapting many techniques known to stabilise generative adversarial networks, and evaluate on a surrogate classification task with the same appearance change. The distilled insights are applied to the problem of free-space segmentation for motion planning in autonomous driving. markus, bewley, ingmar@robots.ox.ac.uk ADAPTED Supervised Mapping Supervised Mapping LabelsTraining Domain Application Domains PredictionsTraining Labels Application Predictions * Performance with available training labels on the target domain. This serves as upper bound for performance.

Learning to Drive from Simulation without Real World Labels

“…The steering limits differ between vehicles, hence we applied a linear calibration to the real vehicle steering output. Adaptation (ADA) [34] to the feature space to align encoders from simulation and real data. For evaluation, we also compared a Drive-Straight policy as a proxy to assess road curvature, and to quantitatively assess the efficacy of offline metrics.…”

Don’t Worry About the Weather: Unsupervised Condition-Dependent Domain Adaptation

“…Domain Confusion: The most common approaches fall under the umbrella of domain confusion, making use of a discriminator that forces features extracted by an encoder to follow a similar distribution for both a source and a target domain [7], [8], [9], [13], [21], [22]. The downside of these approaches is the lack of a direct loss for the target domain, which limits its upper bound on performance.…”

“…Domain unification can be applied within the vast distribution of natural images [1], [2], [3], between natural and synthetic images (computer-generated, whether through traditional 3D rendering or more modern GAN-based techniques) [4], [5] and even between different sensor modalities [6]. Additionally, domain unification can be implemented at different stages of a computer vision pipeline, ranging from direct approaches such as domain confusion [7], [8], [9], fine-tuning models on target domains [1] or mixture-of-expert approaches [10], etc.…”

“…Most approaches attempt to solve this shortcoming by using 3D-rendered simulations that Authors are from the Oxford Robotics Institute, University of Oxford, UK. {horia, tombruls, pnewman}@robots.ox.ac.uk programatically provide ground-truth [11], [12], or by using unsupervised techniques that adapt models based on auxiliary or proxy tasks [7], [8], [9], [13], [1].…”

I Can See Clearly Now: Image Restoration via De-Raining

“…In this paper we choose to use segmentation as an example task by which to test the effectiveness of our method. Many approaches so far have reached for multi-modal data [5], domain adaptation [6], [7] or training on synthetic data [8], however this can become awkward as: 1) Acquiring rainy images is time-consuming, expensive or impossible for many tasks or setups, especially in the case of supervised training, where ground truth data is needed. 2) Training, domain-adapting or fine-tuning each individual task with augmented data is intractable.…”