Towards Understanding Limitations of Pixel Discretization Against Adversarial Attacks

Chen, Jiefeng; Wu, Xi; Rastogi, Vaibhav; Liang, Yingyu; Jha, Somesh

doi:10.1109/eurosp.2019.00042

Cited by 25 publications

(36 citation statements)

References 26 publications

(42 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Edizel et al (2019) attempt to learn typo-resistant word embeddings, but focus on common typos, rather than worst-case typos. In computer vision, Chen et al (2019) discretizes pixels to compute exact robust accuracy on MNIST, but their approach generalizes poorly to other tasks like CIFAR-10. Garg et al (2018) generate functions that map to robust features, while enforcing variation in outputs.…”

Section: Discussionmentioning

confidence: 99%

Robust Encodings: A Framework for Combating Adversarial Typos

Jones

Jia

Raghunathan

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

Despite excellent performance on many tasks, NLP systems are easily fooled by small adversarial perturbations of inputs. Existing procedures to defend against such perturbations are either (i) heuristic in nature and susceptible to stronger attacks or (ii) provide guaranteed robustness to worst-case attacks, but are incompatible with state-of-the-art models like BERT. In this work, we introduce robust encodings (RobEn): a simple framework that confers guaranteed robustness, without making compromises on model architecture. The core component of RobEn is an encoding function, which maps sentences to a smaller, discrete space of encodings. Systems using these encodings as a bottleneck confer guaranteed robustness with standard training, and the same encodings can be used across multiple tasks. We identify two desiderata to construct robust encoding functions: perturbations of a sentence should map to a small set of encodings (stability), and models using encodings should still perform well (fidelity). We instantiate RobEn to defend against a large family of adversarial typos. Across six tasks from GLUE, our instantiation of RobEn paired with BERT achieves an average robust accuracy of 71.3% against all adversarial typos in the family considered, while previous work using a typo-corrector achieves only 35.3% accuracy against a simple greedy attack.

show abstract

Section: Discussionmentioning

confidence: 99%

Robust Encodings: A Framework for Combating Adversarial Typos

Jones

Jia

Raghunathan

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…Secondly, we evaluate the improvement of robustness via SSR. For the baseline methods, we include Madry's training [26] and IG-NORM [10], an recent proposed regularization to improve the robustness of Integrated Gradient.…”

Section: Experiments Ii: Robustness Via Regularizationmentioning

confidence: 99%

“…Towards robustness of attribution map, Singh et al [38] proposes soft margin loss to improve the alignment for attributions. Chen et al [10] propose two regularizations so that nearby points have similar Integrated Gradient. On the other hand, an ad-hoc regularization can be an extra budget for people with pretrained models.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Smoothed Geometry for Robust Attribution

Wang,

Ramkumar

et al. 2020

Preprint

View full text Add to dashboard Cite

Feature attributions are a popular tool for explaining the behavior of Deep Neural Networks (DNNs), but have recently been shown to be vulnerable to attacks that produce divergent explanations for nearby inputs. This lack of robustness is especially problematic in high-stakes applications where adversarially-manipulated explanations could impair safety and trustworthiness. Building on a geometric understanding of these attacks presented in recent work, we identify Lipschitz continuity conditions on models' gradient that lead to robust gradient-based attributions, and observe that smoothness may also be related to the ability of an attack to transfer across multiple attribution methods. To mitigate these attacks in practice, we propose an inexpensive regularization method that promotes these conditions in DNNs, as well as a stochastic smoothing technique that does not require re-training. Our experiments on a range of image models demonstrate that both of these mitigations consistently improve attribution robustness, and confirm the role that smooth geometry plays in these attacks on real, large-scale models.

show abstract

“…A recent line of works [3,38,1,8,9,10,30,32,33,34,40,45,46] studies a class of attacks on machine learning systems commonly known as adversarial examples or evasion attacks. Such attacks target a classifier C trained on problem Π.…”

Section: Introductionmentioning

confidence: 99%

Adversarial Examples and Metrics

Döttling,

Grosse,

Backes

et al. 2020

Preprint

View full text Add to dashboard Cite

Adversarial examples are a type of attack on machine learning (ML) systems which cause misclassification of inputs. Achieving robustness against adversarial examples is crucial to apply ML in the real world. While most prior work on adversarial examples is empirical, a recent line of work establishes fundamental limitations of robust classification based on cryptographic hardness. Most positive and negative results in this field however assume that there is a fixed target metric which constrains the adversary, and we argue that this is often an unrealistic assumption. In this work we study the limitations of robust classification if the target metric is uncertain. Concretely, we construct a classification problem, which admits robust classification by a small classifier if the target metric is known at the time the model is trained, but for which robust classification is impossible for small classifiers if the target metric is chosen after the fact. In the process, we explore a novel connection between hardness of robust classification and bounded storage model cryptography.Preprint. Under review.

show abstract

Towards Understanding Limitations of Pixel Discretization Against Adversarial Attacks

Cited by 25 publications

References 26 publications

Robust Encodings: A Framework for Combating Adversarial Typos

Robust Encodings: A Framework for Combating Adversarial Typos

Smoothed Geometry for Robust Attribution

Adversarial Examples and Metrics

Contact Info

Product

Resources

About