Robust Semantic Segmentation by Redundant Networks With a Layer-Specific Loss Contribution and Majority Vote

Bär, Andreas; Klingner, Marvin; Varghese, Serin; Hüger, Fabian; Schlicht, Peter; Fingscheidt, Tim

doi:10.1109/cvprw50498.2020.00174

Cited by 19 publications

(5 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are several works that attempt to increase robustness for semantic segmentation. [1] proposes a specialized student-teacher architecture for robust semantic segmentation. [18] relies on increasing shape bias of networks to build the robust semantic segmentation system, inspired by the success of image classification using a similar bias approach [9].…”

Section: Related Work and Discussionmentioning

confidence: 99%

“…The other approach is to apply an existing robust data augmentation technique during transfer learning. While applying robustification techniques during finetuning for downstream tasks is an option, a naive application of these methods can decrease downstream task performances 1 and often requires further modifications tailored for downstream tasks to maintain good accuracy while achieving robustness [6], partly because object detection and semantic segmentation systems tend to be more com- 1 See Table 4 in the Appendix as an example plex than image classification. Therefore, rather than entirely resorting to data augmentation during fine-tuning, it is critical to better understand robustness transfer to achieve both robustness and good clean accuracy in downstream tasks.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Does Robustness on ImageNet Transfer to Downstream Tasks?

Yamada¹,

Otani²

2022

Preprint

View full text Add to dashboard Cite

As clean ImageNet accuracy nears its ceiling, the research community is increasingly more concerned about robust accuracy under distributional shifts. While a variety of methods have been proposed to robustify neural networks, these techniques often target models trained on ImageNet classification. At the same time, it is a common practice to use ImageNet pretrained backbones for downstream tasks such as object detection, semantic segmentation, and image classification from different domains. This raises a question: Can these robust image classifiers transfer robustness to downstream tasks? For object detection and semantic segmentation, we find that a vanilla Swin Transformer, a variant of Vision Transformer tailored for dense prediction tasks, transfers robustness better than Convolutional Neural Networks that are trained to be robust to the corrupted version of ImageNet. For CIFAR10 classification, we find that models that are robustified for ImageNet do not retain robustness when fully fine-tuned. These findings suggest that current robustification techniques tend to emphasize ImageNet evaluations. Moreover, network architecture is a strong source of robustness when we consider transfer learning.

show abstract

Section: Related Work and Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Does Robustness on ImageNet Transfer to Downstream Tasks?

Yamada¹,

Otani²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Among online-capable metrics for the overall performance prediction, some only focus at malfunction detection and correction (again involving an ensemble of DNNs) [17], [64], or exploit temporal inconsistency between consecutive predictions [15], which has to be defined in a highly task-specific way. The closest prior work to ours is presumably from Löhdefink et al [18], who propose to train an autoencoder to reconstruct an image on the same data a semantic segmentation DNN is trained on, showing a correlation between both task's metrics.…”

Section: Performance Prediction Of Neural Networkmentioning

confidence: 99%

“…one typically assumes that an offline-measured performance of a DNN is also valid in inference, this is actually not true due to the mentioned environment changes. Meanwhile, less-frequently proposed online-capable algorithms are either task-specific [15], rely on ensembles of DNNs [16], [17], or only show the correlation of a proposed metric to the absolute performance metric without further outlining an online-capable predictive scheme [15], [18]. Naively using the confidence scores of the network itself [19] is not recommended as DNNs often assign a probability of close to one to a single class [20], and even more important, the uncertainty of measurements does not bear predictive power to estimate the absolute DNN performance.…”

mentioning

confidence: 99%

Online Performance Prediction of Perception DNNs by Multi-Task Learning With Depth Estimation

Klingner

Fingscheidt

2021

IEEE Trans. Intell. Transport. Syst.

Self Cite

View full text Add to dashboard Cite

Online performance prediction (or: observation) of deep neural networks (DNNs) in highly automated driving presents an unsolved task until now, as most DNNs are evaluated offline requiring datasets with ground truth labels. In practice, however, DNN performance depends on the used camera type, lighting and weather conditions, and on various other kinds of domain shift. Also, the input to DNN-based perception systems can be perturbed by adversarial attacks requiring means to detect these input perturbations. In this work we propose a method to mitigate these problems by a multi-task learning approach with monocular depth estimation as a secondary task, which enables us to predict the DNN's performance for various other (primary) tasks by evaluating only the depth estimation task with a physical depth measurement provided, e.g., by a LiDAR sensor. We show the effectiveness of our method for the primary task of semantic segmentation using various training datasets, test datasets, model architectures, and input perturbations. Our method provides an effective way to predict (observe) the performance of DNNs for semantic segmentation even on a single-image basis and is transferable to other primary DNN-based perception tasks in a straightforward manner.

show abstract

“…An alternative robust training procedure uses a redundant Teacher-Student framework consisting of three networks named the static teacher, the static student, and the adaptive student [21,22]. The two students apply model distillation of the teacher by learning to predict its output, while having a considerably simpler architecture.…”

Section: Empirically Justifiedmentioning

confidence: 99%

How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review

Tambon,

Laberge,

et al. 2021

Preprint

View full text Add to dashboard Cite

Context: Machine Learning (ML) has been at the heart of many innovations over the past years. However, including it in so-called "safety-critical" systems such as automotive or aeronautic has proven to be very challenging, since the shift in paradigm that ML brings completely changes traditional certification approaches.Objective: This paper aims to elucidate challenges related to the certification of ML-based safety-critical systems, as well as the solutions that are proposed in the literature to tackle them, answering the question "How to Certify Machine Learning Based Safety-critical Systems?".Method: We conduct a Systematic Literature Review (SLR) of research papers published between 2015 to 2020, covering topics related to the certification of ML systems. In total, we identified 217 papers covering topics considered to be the main pillars of ML certification: Robustness, Uncertainty, Explainability, Verification, Safe Reinforcement Learning, and Direct Certification. We analyzed the main trends and problems of each sub-field and provided summaries of the papers extracted.Results: The SLR results highlighted the enthusiasm of the community for this subject, as well as the lack of diversity in term of datasets and type of ML models. It also emphasized the need to further develop connections between academia and industries to deepen the domain study. Finally, it also illustrated This work is supported by the DEEL project CRDPJ 537462-18 funded by the National Science and Engineering Research Council of Canada (NSERC) and the Consortium for Research and Innovation in Aerospace in Québec (CRIAQ), together with its industrial partners Thales Canada inc, Bell Textron Canada Limited, CAE inc and Bombardier inc.

show abstract

Robust Semantic Segmentation by Redundant Networks With a Layer-Specific Loss Contribution and Majority Vote

Cited by 19 publications

References 54 publications

Does Robustness on ImageNet Transfer to Downstream Tasks?

Does Robustness on ImageNet Transfer to Downstream Tasks?

Online Performance Prediction of Perception DNNs by Multi-Task Learning With Depth Estimation

How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review

Contact Info

Product

Resources

About