Adversarial Training Can Hurt Generalization

Raghunathan, Aditi; Xie, Sang Michael; Yang, Fanny; Duchi, John C.; Liang, Percy

doi:10.48550/arxiv.1906.06032

Cited by 76 publications

(74 citation statements)

References 12 publications

(25 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The AdvProp is unique in that successfully aims at performance gain for large-scale models on clean images with adversarial examples. This is different from adversarial training, which generally results in sacrificing model accuracy on the clean images to gain robustness to adversarial examples [2], [282]. There are also other instances that report performance gain on clean data by accounting for adversarial image in training.…”

Section: A Improving Model Performancementioning

confidence: 99%

Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey

2018

View full text Add to dashboard Cite

Deep learning is at the heart of the current rise of artificial intelligence. In the field of Computer Vision, it has become the workhorse for applications ranging from self-driving cars to surveillance and security. Whereas deep neural networks have demonstrated phenomenal success (often beyond human capabilities) in solving complex problems, recent studies show that they are vulnerable to adversarial attacks in the form of subtle perturbations to inputs that lead a model to predict incorrect outputs. For images, such perturbations are often too small to be perceptible, yet they completely fool the deep learning models. Adversarial attacks pose a serious threat to the success of deep learning in practice. This fact has recently lead to a large influx of contributions in this direction. This article presents the first comprehensive survey on adversarial attacks on deep learning in Computer Vision. We review the works that design adversarial attacks, analyze the existence of such attacks and propose defenses against them. To emphasize that adversarial attacks are possible in practical conditions, we separately review the contributions that evaluate adversarial attacks in the real-world scenarios. Finally, drawing on the reviewed literature, we provide a broader outlook of this research direction.

show abstract

Section: A Improving Model Performancementioning

confidence: 99%

Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey

2018

View full text Add to dashboard Cite

show abstract

“…that high standard accuracy is fundamentally at odds with high robust accuracy by considering classification problems, whereas Nakkiran (2019) suggests an alternative explanation that classifiers that are simultaneously robust and accurate are complex, and may not be contained in current function classes. However, Raghunathan et al (2019) shows that the tradeoff is not due to optimization or representation issues by showing that such tradeoffs exist even for a problem with a convex loss where the optimal predictor achieves 100% standard and robust accuracy. In contrast to previous work, we provide sharp and interpretable characterizations of the robustness-accuracy tradeoffs that may arise in regression problems, albeit restricted to linear models.…”

Section: Related Workmentioning

confidence: 99%

“…However, it was soon noticed that while adversarial training could be used to improve model robustness, it often came with a corresponding decrease in accuracy on nominal (unperturbed) data. Further, various simplified theoretical models (Tsipras et al, 2018;Zhang et al, 2019;Nakkiran, 2019;Raghunathan et al, 2019;Chen et al, 2020;Javanmard et al, 2020), have been used to explain this phenomena, and to argue that such robustness-accuracy tradeoffs are unavoidable.…”

Section: Introductionmentioning

confidence: 99%

Adversarial Tradeoffs in Robust State Estimation

Lee¹,

Zhang²,

Hassani³

et al. 2021

Preprint

View full text Add to dashboard Cite

Adversarially robust training has been shown to reduce the susceptibility of learned models to targeted input data perturbations. However, it has also been observed that such adversarially robust models suffer a degradation in accuracy when applied to unperturbed data sets, leading to a robustness-accuracy tradeoff. In this paper, we provide sharp and interpretable characterizations of such robustness-accuracy tradeoffs for linear inverse problems. In particular, we provide an algorithm to find the optimal adversarial perturbation given data, and develop tight upper and lower bounds on the adversarial loss in terms of the standard (non-adversarial) loss and the spectral properties of the resulting estimator. Further, motivated by the use of adversarial training in reinforcement learning, we define and analyze the adversarially robust Kalman Filtering problem. We apply a refined version of our general theory to this problem, and provide the first characterization of robustness-accuracy tradeoffs in a setting where the data is generated by a dynamical system. In doing so, we show a natural connection between a filter's robustness to adversarial perturbation and underlying control theoretic properties of the system being observed, namely the spectral properties of its observability gramian.

show abstract

“…Previously, some works tried to analyze adversarial training from robust optimization [27], robustness generalization [21,32], training strategy [4,15,19,29,34]. However, in these works, they all focus on the averaged robustness over all classes while ignoring the possible difference among different classes.…”

Section: Introductionmentioning

confidence: 99%

Analysis and Applications of Class-wise Robustness in Adversarial Training

Tian¹,

Kuang

Jiang

et al. 2021

Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery &Amp; Data Mining

View full text Add to dashboard Cite

Adversarial training is one of the most effective approaches to improve model robustness against adversarial examples. However, previous works mainly focus on the overall robustness of the model, and the in-depth analysis on the role of each class involved in adversarial training is still missing. In this paper, we propose to analyze the class-wise robustness in adversarial training. First, we provide a detailed diagnosis of adversarial training on six benchmark datasets, i.e., MNIST, CIFAR-10, CIFAR-100, SVHN, STL-10 and ImageNet. Surprisingly, we find that there are remarkable robustness discrepancies among classes, leading to unbalance/unfair class-wise robustness in the robust models. Furthermore, we keep investigating the relations between classes and find that the unbalanced class-wise robustness is pretty consistent among different attack and defense methods. Moreover, we observe that the stronger attack methods in adversarial learning achieve performance improvement mainly from a more successful attack on the vulnerable classes (i.e., classes with less robustness). Inspired by these interesting findings, we design a simple but effective attack method based on the traditional PGD attack, named Temperature-PGD attack, which proposes to enlarge the robustness disparity among classes with a temperature factor on the confidence distribution of each image. Experiments demonstrate our method can achieve a higher attack rate than the PGD attack. Furthermore, from the defense perspective, we also make some modifications in the training and inference phases to improve the robustness of the most vulnerable class, so as to mitigate the large difference in class-wise robustness. We believe our work can contribute to a more comprehensive understanding of adversarial training as well as rethinking the class-wise properties in robust models.

show abstract

Adversarial Training Can Hurt Generalization

Cited by 76 publications

References 12 publications

Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey

Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey

Adversarial Tradeoffs in Robust State Estimation

Analysis and Applications of Class-wise Robustness in Adversarial Training

Contact Info

Product

Resources

About