Benchmarking the Robustness of Semantic Segmentation Models with Respect to Common Corruptions

Kamann, Christoph; Rother, Carsten

doi:10.1007/s11263-020-01383-2

Cited by 39 publications

(26 citation statements)

References 67 publications

(68 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Further experiments also distort the test samples to see the effect of each distortion on the performance. The chosen distortions were based on [23]. Before discussing the results, the dataset and distortions are explained in detail.…”

Section: Experiments and Resultsmentioning

confidence: 99%

Selective Probabilistic Classifier Based on Hypothesis Testing

Germi

Rahtu

Huttunen³

2021

2021 9th European Workshop on Visual Information Processing (EUVIP)

View full text Add to dashboard Cite

In this paper, we propose a simple yet effective method to deal with the violation of the Closed-World Assumption for a classifier. Previous works tend to apply a threshold either on the classification scores or the loss function to reject the inputs that violate the assumption. However, these methods cannot achieve the low False Positive Ratio (FPR) required in safety applications. The proposed method is a rejection option based on hypothesis testing with probabilistic networks. With probabilistic networks, it is possible to estimate the distribution of outcomes instead of a single output. By utilizing Z-test over the mean and standard deviation for each class, the proposed method can estimate the statistical significance of the network certainty and reject uncertain outputs. The proposed method was experimented on with different configurations of the COCO and CIFAR datasets. The performance of the proposed method is compared with the Softmax Response, which is a known top-performing method. It is shown that the proposed method can achieve a broader range of operation and cover a lower FPR than the alternative.

show abstract

Section: Experiments and Resultsmentioning

confidence: 99%

Selective Probabilistic Classifier Based on Hypothesis Testing

Germi

Rahtu

Huttunen³

2021

2021 9th European Workshop on Visual Information Processing (EUVIP)

View full text Add to dashboard Cite

show abstract

“…A simple way to improve the robustness of a machine learning model is data augmentation [61]. However, even though corrupted data can enhance the robustness of CNN models in semantic segmentation [62], introducing corrupted data into the training domain largely diminishes the model accuracy on uncorrupted data.…”

Section: Model Robustnessmentioning

confidence: 99%

Robust Semantic Segmentation With Multi-Teacher Knowledge Distillation

Amirkhani¹,

Khosravian

Masih‐Tehrani

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Recent studies have recently exploited knowledge distillation (KD) technique to address timeconsuming annotation task in semantic segmentation, through which one teacher trained on a single dataset could be leveraged for annotating unlabeled data. However, in this context, knowledge capacity is restricted, and knowledge variety is rare in different conditions, such as cross-model KD, in which the single teacher KD prohibits the student model from distilling information using cross-domain context. To fix this concern, we have looked into learning a lightweight student from a group of teachers. To be more specific, we train five distinct lightweight convolutional neural networks (CNNs) for semantic segmentation on several datasets. Several state-of-the-art augmentation transformations have also been utilized in our training phase. The impacts of such training scenarios are then assessed in terms of student robustness and accuracy. As the main contribution of this paper, our proposed multi-teacher KD paradigm endows the student with the ability to amalgamate and capture a variety of knowledge illustrations from different sources. Results demonstrated that our method outperforms the existing studies on both clean and corrupted data in the semantic segmentation task while benefiting from our proposed score weight system. Experiments validate that our multi-teacher framework results in an improvement of 9% up to 32.18% compared to the singleteacher paradigm. Moreover, it is demonstrated that our paradigm surpasses previous supervised real-time studies in the semantic segmentation challenge.

show abstract

“…While CNNs focus on textures [20], Transformers put more importance on the object shape [3,55], which is more similar to human vision [20]. For semantic segmentation, ASPP [7] and skip connections [60] were reported to increase the robustness [36]. Further, Xie et al [86] showed that their Transformer-based architecture improves the robustness over CNN-based networks.…”

Section: Related Workmentioning

confidence: 99%

“…Previous works on semantic segmentation with Transformer backbones usually exploit only local information for the decoder [80,86,96]. In contrast, we propose to utilize additional context information in the decoder as this has been shown to increase the robustness of semantic segmentation [36], a helpful property for UDA. Instead of just considering the context information of the bottleneck features [6,7], DAFormer uses the context across features from different encoder levels as the additional earlier features provide valuable low-level concepts for semantic segmentation at a high resolution, which can also provide important context information.…”

Section: Daformer Network Architecturementioning

confidence: 99%

DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation

Hoyer¹,

Dai²,

Gool³

2021

Preprint

View full text Add to dashboard Cite

As acquiring pixel-wise annotations of real-world images for semantic segmentation is a costly process, a model can instead be trained with more accessible synthetic data and adapted to real images without requiring their annotations. This process is studied in unsupervised domain adaptation (UDA). Even though a large number of methods propose new adaptation strategies, they are mostly based on outdated network architectures. As the influence of recent network architectures has not been systematically studied, we first benchmark different network architectures for UDA and then propose a novel UDA method, DAFormer, based on the benchmark results. The DAFormer network consists of a Transformer encoder and a multi-level contextaware feature fusion decoder. It is enabled by three simple but crucial training strategies to stabilize the training and to avoid overfitting DAFormer to the source domain: While the Rare Class Sampling on the source domain improves the quality of pseudo-labels by mitigating the confirmation bias of self-training towards common classes, the Thing-Class ImageNet Feature Distance and a learning rate warmup promote feature transfer from ImageNet pretraining. DAFormer significantly improves the state-of-the-art performance by 10.8 mIoU for GTA→Cityscapes and 5.4 mIoU for Synthia→Cityscapes and enables learning even difficult classes such as train, bus, and truck well. The implementation is available at https://github.com/ lhoyer/DAFormer.

show abstract

Benchmarking the Robustness of Semantic Segmentation Models with Respect to Common Corruptions

Cited by 39 publications

References 67 publications

Selective Probabilistic Classifier Based on Hypothesis Testing

Selective Probabilistic Classifier Based on Hypothesis Testing

Robust Semantic Segmentation With Multi-Teacher Knowledge Distillation

DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation

Contact Info

Product

Resources

About