Robustness via Cross-Domain Ensembles

Yeo, Teresa; Kar, Oğuzhan Fatih; Zamir, Amir

doi:10.1109/iccv48922.2021.01197

Cited by 15 publications

(5 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This also follows intuition: the representation required for semantic segmentation is a subset of the representation required for part segmentation. This observation is also consistent with recent multi-task learning frameworks [47,45]. Task relationships are dynamic.…”

Section: Understanding the Structure Of Taskssupporting

confidence: 91%

Auto-Lambda: Disentangling Dynamic Task Relationships

Liu¹,

James²,

Davison³

et al. 2022

Preprint

View full text Add to dashboard Cite

Understanding the structure of multiple related tasks allows for multi-task learning to improve the generalisation ability of one or all of them. However, it usually requires training each pairwise combination of tasks together in order to capture task relationships, at an extremely high computational cost. In this work, we learn task relationships via an automated weighting framework, named Auto-λ. Unlike previous methods where task relationships are assumed to be fixed, Auto-λ is a gradient-based meta learning framework which explores continuous, dynamic task relationships via task-specific weightings, and can optimise any choice of combination of tasks through the formulation of a meta-loss; where the validation loss automatically influences task weightings throughout training. We apply the proposed framework to both multi-task and auxiliary learning problems in computer vision and robotics, and show that Auto-λ achieves state-of-the-art performance, even when compared to optimisation strategies designed specifically for each problem and data domain. Finally, we observe that Auto-λ can discover interesting learning behaviors, leading to new insights in multi-task learning. Code is available at https://github.com/lorenmt/auto-lambda.

show abstract

Section: Understanding the Structure Of Taskssupporting

confidence: 91%

Auto-Lambda: Disentangling Dynamic Task Relationships

Liu¹,

James²,

Davison³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…While the standard recipe for creating model ensembles is based on training multiple identical models from different random initializations [LPB17], there do exist other methods for introducing diversity. Examples include training models with different hyperparameters [WST+20], data augmentations [SM20], input transformations [YKZ21], or model architectures [ZZE+20]. Note that, in contrast to our work, none of these approaches incorporate this diversity into training itself.…”

Section: Consistency Regularizationmentioning

confidence: 95%

Combining Diverse Feature Priors

Jain¹,

Tsipras²,

Mądry³

2021

Preprint

View full text Add to dashboard Cite

To improve model generalization, model designers often restrict the features that their models use, either implicitly or explicitly. In this work, we explore the design space of leveraging such feature priors by viewing them as distinct perspectives on the data. Specifically, we find that models trained with diverse sets of feature priors have less overlapping failure modes, and can thus be combined more effectively. Moreover, we demonstrate that jointly training such models on additional (unlabeled) data allows them to correct each other's mistakes, which, in turn, leads to better generalization and resilience to spurious correlations. 1

show abstract

“…Reducing the adversarial transferability among base models in an ensemble can achieve good robustness without sacrificing benign accuracy [70][71][72]. To further verify the differences between ResNet and ViT in the context of remote sensing, we found empirical evidence that the adversarial examples of RSIs tend to have weak transferability between CNNs and ViTs, which facilities constructing ensemble classifiers to generate a more robust model.…”

Section: Weak Adversarial Transferabilitymentioning

confidence: 99%

Adversarial Robustness Enhancement of UAV-Oriented Automatic Image Recognition Based on Deep Ensemble Models

Sun

2023

Remote Sensing

View full text Add to dashboard Cite

Deep neural networks (DNNs) have been widely utilized in automatic visual navigation and recognition on modern unmanned aerial vehicles (UAVs), achieving state-of-the-art performances. However, DNN-based visual recognition systems on UAVs show serious vulnerability to adversarial camouflage patterns on targets and well-designed imperceptible perturbations in real-time images, which poses a threat to safety-related applications. Considering a scenario in which a UAV is suffering from adversarial attack, in this paper, we investigate and construct two ensemble approaches with CNN and transformer for both proactive (i.e., generate robust models) and reactive (i.e., adversarial detection) adversarial defense. They are expected to be secure under attack and adapt to the resource-limited environment on UAVs. Specifically, the probability distributions of output layers from base DNN models in the ensemble are combined in the proactive defense, which mainly exploits the weak adversarial transferability between the CNN and transformer. For the reactive defense, we integrate the scoring functions of several adversarial detectors with the hidden features and average the output confidence scores from ResNets and ViTs as a second integration. To verify their effectiveness in the recognition task of remote sensing images, we conduct experiments on both optical and synthetic aperture radar (SAR) datasets. We find that the ensemble model in proactive defense performs as well as three popular counterparts, and both of the ensemble approaches can achieve much more satisfactory results than a single base model/detector, which effectively alleviates adversarial vulnerability without extra re-training. In addition, we establish a one-stop platform for conveniently evaluating adversarial robustness and performing defense on recognition models called AREP-RSIs, which is beneficial for the future research of the remote sensing field.

show abstract

Robustness via Cross-Domain Ensembles

Cited by 15 publications

References 13 publications

Auto-Lambda: Disentangling Dynamic Task Relationships

Auto-Lambda: Disentangling Dynamic Task Relationships

Combining Diverse Feature Priors

Adversarial Robustness Enhancement of UAV-Oriented Automatic Image Recognition Based on Deep Ensemble Models

Contact Info

Product

Resources

About