DeepSearch: a simple and effective blackbox attack for deep neural networks

Zhang, Fuyuan; Chowdhury, Sankalan Pal; Christakis, Maria

doi:10.1145/3368089.3409750

Cited by 33 publications

(20 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Subtle imperceptible perturbations of inputs, known as adversarial examples, can change their prediction results. Various algorithms [Carlini and Wagner 2017b;Goodfellow et al 2015;Madry et al 2018;Tabacof and Valle 2016;Zhang et al 2020] have been proposed that can effectively find adversarial examples. Research on developing defense mechanisms against adversarial examples Wagner 2016, 2017a,b;Cornelius 2019;Engstrom et al 2018;Goodfellow et al 2015;Huang et al 2015;Mirman et al , 2019 is also active.…”

Section: Robustness Of Deep Neuralmentioning

confidence: 99%

See 1 more Smart Citation

Perfectly parallel fairness certification of neural networks

Urban

Christakis

Wüstholz³

et al. 2020

Proc. ACM Program. Lang.

Self Cite

View full text Add to dashboard Cite

Recently, there is growing concern that machine-learned software, which currently assists or even automates decision making, reproduces, and in the worst case reinforces, bias present in the training data. The development of tools and techniques for certifying fairness of this software or describing its biases is, therefore, critical. In this paper, we propose a perfectly parallel static analysis for certifying fairness of feed-forward neural networks used for classification of tabular data. When certification succeeds, our approach provides definite guarantees, otherwise, it describes and quantifies the biased input space regions. We design the analysis to be sound, in practice also exact, and configurable in terms of scalability and precision, thereby enabling pay-as-you-go certification. We implement our approach in an open-source tool called libra and demonstrate its effectiveness on neural networks trained on popular datasets. CCS Concepts: • Software and its engineering → Formal software verification; • Theory of computation → Program analysis; Abstraction; • Computing methodologies → Neural networks; • Social and professional topics → Computing / technology policy.

show abstract

Section: Robustness Of Deep Neuralmentioning

confidence: 99%

“…Odena et al [Odena et al 2019] were the first to develop coverage-guided fuzzing for neural networks. Zhang et al [Zhang et al 2020] proposed a blackbox-fuzzing technique to test their robustness.…”

Section: Robustness Of Deep Neuralmentioning

confidence: 99%

Perfectly parallel fairness certification of neural networks

Urban

Christakis

Wüstholz³

et al. 2020

Proc. ACM Program. Lang.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The main property of adversarially perturbed images is that they are very close from a human point of view to the original benign image, but trick the model into predicting the wrong class. While the notion of "closeness from a human perspective" is hard to quantify, there exists general consensus around using the L 0 , L 2 , and L ∞ norms as proxies for measuring the adversarial perturbations [31,23,22,11,32].…”

Section: Introductionmentioning

confidence: 99%

“…EvoBA focuses on the L 0 norm, is fast, query-efficient, and effective. Therefore, we propose using it together with similarly fast and efficient methods that focus on different norms, such as SimBA, which focuses on L 2 , and DeepSearch ( [32]), which focuses on L ∞ , to empirically evaluate the robustness of image classifiers. These methods can act together as a fast and general toolbox used along the way of developing systems involving image classification models.…”

Section: Introductionmentioning

confidence: 99%

EvoBA: An Evolution Strategy as a Strong Baseline forBlack-Box Adversarial Attacks

Ilie,

Popescu,

Stefanescu

2021

Preprint

View full text Add to dashboard Cite

Recent work has shown how easily white-box adversarial attacks can be applied to state-of-the-art image classifiers. However, real-life scenarios resemble more the black-box adversarial conditions, lacking transparency and usually imposing natural, hard constraints on the query budget. We propose EvoBA 1 , a black-box adversarial attack based on a surprisingly simple evolutionary search strategy. EvoBA is query-efficient, minimizes L0 adversarial perturbations, and does not require any form of training. EvoBA shows efficiency and efficacy through results that are in line with much more complex state-of-the-art black-box attacks such as AutoZOOM. It is more query-efficient than SimBA, a simple and powerful baseline black-box attack, and has a similar level of complexity. Therefore, we propose it both as a new strong baseline for black-box adversarial attacks and as a fast and general tool for gaining empirical insight into how robust image classifiers are with respect to L0 adversarial perturbations. There exist fast and reliable L2 black-box attacks, such as SimBA, and L∞ blackbox attacks, such as DeepSearch. We propose EvoBA as a query-efficient L0 black-box adversarial attack which, together with the aforementioned methods, can serve as a generic tool to assess the empirical robustness of image classifiers. The main advantages of such methods are that they run fast, are query-efficient, and can easily be integrated in image classifiers development pipelines. While our attack minimises the L0 adversarial perturbation, we also report L2, and notice that we compare favorably to the state-of-the-art L2 black-box attack, AutoZOOM, and of the L2 strong baseline, SimBA.

show abstract

“…Odena et al[24] first design a coverageguided fuzzing framework for testing deep neural networks. Zhang et al[28] develop a fuzzing-based blackbox adversarial attack for DNNs. Moreover, Khmelnitsky et al[9] design an technique to extract a surrogate automata model to analyze and verify regular properties for recurrent neural networks.…”

mentioning

confidence: 99%

DeepGalaxy: Testing Neural Network Verifiers via Two-Dimensional Input Space Exploration

Xie¹,

Zhang²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Deep neural networks (DNNs) are widely developed and applied in many areas, and the quality assurance of DNNs is critical. Neural network verification (NNV) aims to provide formal guarantees to DNN models. Similar to traditional software, neural network verifiers could also contain bugs, which would have a critical and serious impact, especially in safetycritical areas. However, little work exists on validating neural network verifiers. In this work, we propose DeepGalaxy, an automated approach based on differential testing to tackle this problem. Specifically, we (1) propose a line of mutation rules, including model level mutation and specification level mutation, to effectively explore the two-dimensional input space of neural network verifiers; and (2) propose heuristic strategies to select test cases. We leveraged our implementation of DeepGalaxy to test three state-of-the-art neural network verifies, Marabou, Eran, and Neurify. The experimental results support the efficiency and effectiveness of DeepGalaxy. Moreover, five unique unknown bugs were discovered.

show abstract

DeepSearch: a simple and effective blackbox attack for deep neural networks

Cited by 33 publications

References 51 publications

Perfectly parallel fairness certification of neural networks

Perfectly parallel fairness certification of neural networks

EvoBA: An Evolution Strategy as a Strong Baseline forBlack-Box Adversarial Attacks

DeepGalaxy: Testing Neural Network Verifiers via Two-Dimensional Input Space Exploration

Contact Info

Product

Resources

About