A Domain Agnostic Normalization Layer for Unsupervised Adversarial Domain Adaptation

Romijnders, Rob; Meletis, Panagiotis; Dubbelman, Gijs

doi:10.1109/wacv.2019.00203

Cited by 25 publications

(30 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In reality, this assumption rarely holds, but we leave investigation of this matter and how to solve it during inference to future research. Methods that perform domain agnostic inference, like [20], can hold solutions for this problem.…”

Section: Discussion and Future Workmentioning

confidence: 99%

On Boosting Semantic Street Scene Segmentation with Weak Supervision

Meletis

Dubbelman

2019

2019 IEEE Intelligent Vehicles Symposium (IV)

View full text Add to dashboard Cite

Training convolutional networks for semantic segmentation requires per-pixel ground truth labels, which are very time consuming and hence costly to obtain. Therefore, in this work, we research and develop a hierarchical deep network architecture and the corresponding loss for semantic segmentation that can be trained from weak supervision, such as bounding boxes or image level labels, as well as from strong per-pixel supervision. We demonstrate that the hierarchical structure and the simultaneous training on strong (per-pixel) and weak (bounding boxes) labels, even from separate datasets, consistently increases the performance against per-pixel only training. Moreover, we explore the more challenging case of adding weak image-level labels. We collect street scene images and weak labels from the immense Open Images dataset to generate the OpenScapes dataset, and we use this novel dataset to increase segmentation performance on two established per-pixel labeled datasets, Cityscapes and Vistas. We report performance gains up to +13.2% mIoU on crucial street scene classes, and inference speed of 20 fps on a Titan V GPU for Cityscapes at 512 x 1024 resolution. Our network and OpenScapes dataset are shared with the research community.

show abstract

Section: Discussion and Future Workmentioning

confidence: 99%

On Boosting Semantic Street Scene Segmentation with Weak Supervision

Meletis

Dubbelman

2019

2019 IEEE Intelligent Vehicles Symposium (IV)

View full text Add to dashboard Cite

show abstract

“…than the background, Saleh et al propose a simple yet powerful domain generalization segmentation framework by fusing Mask-RCNN's detection results of the foreground [101] and DeepLab's segmentation results of the background [102]. DAN [103]. As normalization (e.g., batch normalization) plays a key role in many neural semantic segmentation networks, DAN improves their performance in the target domain by simply replacing the normalization parameters with the statistics of the target domain.…”

Section: Other Methodsmentioning

confidence: 99%

A Curriculum Domain Adaptation Approach to the Semantic Segmentation of Urban Scenes

Zhang

David

Foroosh

et al. 2020

IEEE Trans. Pattern Anal. Mach. Intell.

130

View full text Add to dashboard Cite

During the last half decade, convolutional neural networks (CNNs) have triumphed over semantic segmentation, which is one of the core tasks in many applications such as autonomous driving and augmented reality. However, to train CNNs requires a considerable amount of data, which is difficult to collect and laborious to annotate. Recent advances in computer graphics make it possible to train CNNs on photo-realistic synthetic imagery with computer-generated annotations. Despite this, the domain mismatch between the real images and the synthetic data hinders the models' performance. Hence, we propose a curriculum-style learning approach to minimizing the domain gap in urban scene semantic segmentation. The curriculum domain adaptation solves easy tasks first to infer necessary properties about the target domain; in particular, the first task is to learn global label distributions over images and local distributions over landmark superpixels. These are easy to estimate because images of urban scenes have strong idiosyncrasies (e.g., the size and spatial relations of buildings, streets, cars, etc.). We then train a segmentation network, while regularizing its predictions in the target domain to follow those inferred properties. In experiments, our method outperforms the baselines on two datasets and two backbone networks. We also report extensive ablation studies about our approach.

show abstract

“…A classbalanced self learning approach is proposed in CBST [20]. Introducing new normalization method, regularization technique, or new loss functions that are specific for domain adaptation problem is invesigated in [21]- [24]. Romijnders et al discuss the limitations of the traditional normalization methods such as batch normalization, and propose a new domain agnostic normalization layer that is more suitable for domain adaptation [21].…”

Section: A Related Workmentioning

confidence: 99%

ColorMapGAN: Unsupervised Domain Adaptation for Semantic Segmentation Using Color Mapping Generative Adversarial Networks

Taşar

Happy

Tarabalka³

et al. 2020

IEEE Trans. Geosci. Remote Sensing

110

View full text Add to dashboard Cite

Due to the various reasons such as atmospheric effects and differences in acquisition, it is often the case that there exists a large difference between spectral bands of satellite images collected from different geographic locations. The large shift between spectral distributions of training and test data causes the current state of the art supervised learning approaches to output unsatisfactory maps. We present a novel semantic segmentation framework that is robust to such shift. The key component of the proposed framework is Color Mapping Generative Adversarial Networks (ColorMapGAN), which can generate fake training images that are semantically exactly the same as training images, but whose spectral distribution is similar to the distribution of the test images. We then use the fake images and the ground-truth for the training images to fine-tune the already trained classifier. Contrary to the existing Generative Adversarial Networks (GANs), the generator in ColorMapGAN does not have any convolutional or pooling layers. It learns to transform the colors of the training data to the colors of the test data by performing only one element-wise matrix multiplication and one matrix addition operations. Thanks to the architecturally simple but powerful design of ColorMapGAN, the proposed framework outperforms the existing approaches with a large margin in terms of both accuracy and computational complexity.

show abstract

A Domain Agnostic Normalization Layer for Unsupervised Adversarial Domain Adaptation

Cited by 25 publications

References 29 publications

On Boosting Semantic Street Scene Segmentation with Weak Supervision

On Boosting Semantic Street Scene Segmentation with Weak Supervision

A Curriculum Domain Adaptation Approach to the Semantic Segmentation of Urban Scenes

ColorMapGAN: Unsupervised Domain Adaptation for Semantic Segmentation Using Color Mapping Generative Adversarial Networks

Contact Info

Product

Resources

About