Unsupervised Domain Adaptation to Improve Image Segmentation Quality Both in the Source and Target Domain

Bolte, Jan-Aike; Kamp, M.; Breuer, Antonia; Homoceanu, Silviu; Schlicht, Peter; Hüger, Fabian; Lipinski, Daniel; Fingscheidt, Tim

doi:10.1109/cvprw.2019.00181

Cited by 52 publications

(30 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For adaptation, we use only test images; for finetune (oracle), we use the labels of train to finetune parameters of pretrained DNN and evaluate on test images. DLV3 with W1 has an mIoU of 60.4%, which is higher than other UDA methods on semantic KITTI ( [25]). Currently, our method achieves the best UDA performance on the benchmark.…”

Section: Methodsmentioning

confidence: 81%

“…The usage of GANs for domain adaptation is applied in semantic segmentation where the discriminators are deployed at various locations of the network to measure the differences between the distributions of the features of source and target domains [22], [23], [24]. Bolte et al [25] use GANs and obtain a single model for both domains; however, the adaptation is applied between datasets with similar appearances under similar weather conditions and all parameters of the network are updated by using the source domain labels. Adaptation is not limited to GANs, Peng et al [26] use Wasserstein GANs (W-GANs) [4] with penalty gradient [27] with an additional generator in the network to measure the distribution similarity.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Semantic Segmentation With Unsupervised Domain Adaptation Under Varying Weather Conditions for Autonomous Vehicles

Erkent

Laugier

2020

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Semantic information provides a valuable source for scene understanding around autonomous vehicles in order to plan their actions and make decisions; however, varying weather conditions reduce the accuracy of the semantic segmentation. We propose a method to adapt to varying weather conditions without supervision, namely without labeled data. We update the parameters of a deep neural network (DNN) model that is pre-trained on the known weather condition (source domain) to adapt it to the new weather conditions (target domain) without forgetting the segmentation in the known weather condition. Furthermore, we don't require the labels from the source domain during adaptation training. The parameters of the DNN are optimized to reduce the distance between the distribution of the features from the images of old and new weather conditions. To measure this distance, we propose three alternatives: W-GAN, GAN and maximum-mean discrepancy (MMD). We evaluate our method on various datasets with varying weather conditions. The results show that the accuracy of the semantic segmentation is improved for varying conditions after adaptation with the proposed method.

show abstract

Section: Methodsmentioning

confidence: 81%

Section: Introductionmentioning

confidence: 99%

Semantic Segmentation With Unsupervised Domain Adaptation Under Varying Weather Conditions for Autonomous Vehicles

Erkent

Laugier

2020

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

show abstract

“…Iverson . (8) with [•] being the Iverson bracket, which is 0 or 1 if the condition inside the bracket is false or true, respectively. Note that the "ground truth" depth maps d obtained from a LiDAR sensor are only sparse, meaning that only for a subset of pixels i ∈ I (d) ⊂ I a depth value d i is available.…”

Section: Performance Evaluation Metricsmentioning

confidence: 99%

“…For example, when the predicted performance is too low, the high-level planning could decide not to trust the current information from the environment perception. This is of special importance when considering that in practice the DNN performance is often very sensitive to changes of the environment, which have not been included into the dataset, the neural network was trained on [7], [8]. Such changes include, e.g., a different camera type, different lighting or weather conditions, various other kinds of domain shift, or even directed adversarial attacks [9], [10], which are difficult to detect on the input image.…”

mentioning

confidence: 99%

Online Performance Prediction of Perception DNNs by Multi-Task Learning With Depth Estimation

Klingner

Fingscheidt

2021

IEEE Trans. Intell. Transport. Syst.

Self Cite

View full text Add to dashboard Cite

Online performance prediction (or: observation) of deep neural networks (DNNs) in highly automated driving presents an unsolved task until now, as most DNNs are evaluated offline requiring datasets with ground truth labels. In practice, however, DNN performance depends on the used camera type, lighting and weather conditions, and on various other kinds of domain shift. Also, the input to DNN-based perception systems can be perturbed by adversarial attacks requiring means to detect these input perturbations. In this work we propose a method to mitigate these problems by a multi-task learning approach with monocular depth estimation as a secondary task, which enables us to predict the DNN's performance for various other (primary) tasks by evaluating only the depth estimation task with a physical depth measurement provided, e.g., by a LiDAR sensor. We show the effectiveness of our method for the primary task of semantic segmentation using various training datasets, test datasets, model architectures, and input perturbations. Our method provides an effective way to predict (observe) the performance of DNNs for semantic segmentation even on a single-image basis and is transferable to other primary DNN-based perception tasks in a straightforward manner.

show abstract

“…For instance, unsupervised domain adaptation was attempted with a domain-adversarial neural network employing a gradient reversal layer during training in [30]. There are indications that this unsupervised domain adaptation approach can improve the segmentation quality in source and target domain [31].…”

Section: Trainingmentioning

confidence: 99%

Automated Quantitative Analyses of Fatigue-Induced Surface Damage by Deep Learning

et al. 2020

View full text Add to dashboard Cite

The digitization of materials is the prerequisite for accelerating product development. However, technologically, this is only beneficial when reliability is maintained. This requires comprehension of the microstructure-driven fatigue damage mechanisms across scales. A substantial fraction of the lifetime for high performance materials is attributed to surface damage accumulation at the microstructural scale (e.g., extrusions and micro crack formation). Although, its modeling is impeded by a lack of comprehensive understanding of the related mechanisms. This makes statistical validation at the same scale by micromechanical experimentation a fundamental requirement. Hence, a large quantity of processed experimental data, which can only be acquired by automated experiments and data analyses, is crucial. Surface damage evolution is often accessed by imaging and subsequent image post-processing. In this work, we evaluated deep learning (DL) methodologies for semantic segmentation and different image processing approaches for quantitative slip trace characterization. Due to limited annotated data, a U-Net architecture was utilized. Three data sets of damage locations observed in scanning electron microscope (SEM) images of ferritic steel, martensitic steel and copper specimens were prepared. In order to allow the developed models to cope with material-specific damage morphology and imaging-induced variance, a customized augmentation pipeline for the input images was developed. Material domain generalizability of ferritic steel and conjunct material trained models were tested successfully. Multiple image processing routines to detect slip trace orientation (STO) from the DL segmented extrusion areas were implemented and assessed. In conclusion, generalization to multiple materials has been achieved for the DL methodology, suggesting that extending it well beyond fatigue damage is feasible.

show abstract

Unsupervised Domain Adaptation to Improve Image Segmentation Quality Both in the Source and Target Domain

Cited by 52 publications

References 32 publications

Semantic Segmentation With Unsupervised Domain Adaptation Under Varying Weather Conditions for Autonomous Vehicles

Semantic Segmentation With Unsupervised Domain Adaptation Under Varying Weather Conditions for Autonomous Vehicles

Online Performance Prediction of Perception DNNs by Multi-Task Learning With Depth Estimation

Automated Quantitative Analyses of Fatigue-Induced Surface Damage by Deep Learning

Contact Info

Product

Resources

About