Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach

Weng, Tsui-Wei; Zhang, Huan; Chen, Pin-Yu; Yi, Jinfeng; Su, Dong; Gao, Yupeng; Hsieh, Cho‐Jui; Daniel, Luca

doi:10.48550/arxiv.1801.10578

Cited by 61 publications

(57 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, several papers consider the question of what assessment methodology and evaluating metrics should be used [38], [41]- [43]. Batsani et al [41] used the distortion of AEs as the robustness metric, while Weng et al [44] proposed a new metric for robustness called CLEVER, based on extreme value theory. Peng et al [45] proposed the EDLIC framework for a quantitative analysis on different threat models and defense techniques.…”

Section: Evaluation Metricsmentioning

confidence: 99%

Evaluation of Neural Networks Defenses and Attacks using NDCG and Reciprocal Rank Metrics

Brama,

Dery,

Grinshpoun

2022

Preprint

View full text Add to dashboard Cite

The problem of attacks on neural networks through input modification (i.e., adversarial examples) has attracted much attention recently. Being relatively easy to generate and hard to detect, these attacks pose a security breach that many suggested defenses try to mitigate. However, the evaluation of the effect of attacks and defenses commonly relies on traditional classification metrics, without adequate adaptation to adversarial scenarios. Most of these metrics are accuracy-based, and therefore may have a limited scope and low distinctive power. Other metrics do not consider the unique characteristics of neural networks functionality, or measure the effect of the attacks indirectly (e.g., through the complexity of their generation). In this paper, we present two metrics which are specifically designed to measure the effect of attacks, or the recovery effect of defenses, on the output of neural networks in multiclass classification tasks. Inspired by the normalized discounted cumulative gain and the reciprocal rank metrics used in information retrieval literature, we treat the neural network predictions as ranked lists of results. Using additional information about the probability of the rank enabled us to define novel metrics that are suited to the task at hand. We evaluate our metrics using various attacks and defenses on a pretrained VGG19 model and the ImageNet dataset. Compared to the common classification metrics, our proposed metrics demonstrate superior informativeness and distinctiveness.

show abstract

Section: Evaluation Metricsmentioning

confidence: 99%

Evaluation of Neural Networks Defenses and Attacks using NDCG and Reciprocal Rank Metrics

Brama,

Dery,

Grinshpoun

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Many studies addressing the formal verification of ANN classifiers have examined the extent to which test instances can be perturbed without yielding a change in the assigned class (Weng, 2018). This seam of research was a response to the observation that images which have been correctly assigned to a particular class by an ANN classifier, are sometimes assigned to an alternative, incorrect class when subject to minor modifications.…”

Section: Related Workmentioning

confidence: 99%

An Integrated Process for Verifying Deep Learning Classifiers Using Dataset Dissimilarity Measures

Hond

Asgari

Jeffery

et al. 2021

International Journal of Artificial Intelligence and Machine Learning

View full text Add to dashboard Cite

The specification and verification of algorithms is vital for safety-critical autonomous systems which incorporate deep learning elements. We propose an integrated process for verifying artificial neural network (ANN) classifiers. This process consists of an off-line verification and an on-line performance prediction phase. The process is intended to verify ANN classifier generalisation performance, and to this end makes use of dataset dissimilarity measures. We introduce a novel measure for quantifying the dissimilarity between the dataset used to train a classification algorithm, and the test dataset used to evaluate and verify classifier performance. A system-level requirement could specify the permitted form of the functional relationship between classifier performance and a dissimilarity measure; such a requirement could be verified by dynamic testing. Experimental results, obtained using publicly available datasets, suggest that the measures have relevance to real-world practice for both quantifying dataset dissimilarity, and specifying and verifying classifier performance.

show abstract

“…Effectively evaluating the adversarial robustness of DNNs is an important issue for both academic research and practical applications. Most of the previous evaluations can be divided into two groups: attack-based methods [4], [6], [10], [11] and bound-based methods [7], [33], [8], [34], [9]. Attack-based evaluations, such as FGSM [4] and PGD [10], directly generate adversarial examples to attack deep models to assess the worst-case risk, i.e., adversarial risk, as a robustness indicator.…”

Section: Robustness Evaluationmentioning

confidence: 99%

“…Bound-based methods try to provide a fundamental bound, or its estimation, for analyzing the robustness of classifiers to adversarial examples. Weng et al [8] proposed a metric named the Cross-Lipschitz extreme value for network robustness (CLEVER) to measure the robustness without any attack. Weng et al [34] provided efficient algorithms, i.e., Fast-Lin and Fast-Lip, for computing a certified lower bound by exploiting the ReLU property.…”

Section: Robustness Evaluationmentioning

confidence: 99%

“…Various methods have been proposed over the last few years [6], [7], [8], [9] to evaluate the adversarial robustness of DNNs. One of the most practical methods [4], [10], [11] is to approximate the adversarial risk of the subjected model f as a robustness indicator, i.e., R(f, D, B, ) = E (x,y)∼D max x ∈B(x, ) 0−1 (f (x ), y) , (1) P. Xia, Z. Li, and B. Li are with the Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei, China (e-mail: {xpengfei, iceli}@mail.ustc.edu.cn, binli@ustc.edu.cn).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Tightening the Approximation Error of Adversarial Risk with Auto Loss Function Search

Xia¹,

Li²,

Li³

2021

Preprint

View full text Add to dashboard Cite

Numerous studies have demonstrated that deep neural networks are easily misled by adversarial examples. Effectively evaluating the adversarial robustness of a model is important for its deployment in practical applications. Currently, a common type of evaluation is to approximate the adversarial risk of a model as a robustness indicator by constructing malicious instances and executing attacks. Unfortunately, there is an error (gap) between the approximate value and the true value. Previous studies manually design attack methods to achieve a smaller error, which is inefficient and may miss a better solution.In this paper, we establish the tightening of the approximation error as an optimization problem and try to solve it with an algorithm. More specifically, we first analyze that replacing the non-convex and discontinuous 0-1 loss with a surrogate loss, a necessary compromise in calculating the approximation, is one of the main reasons for the error. Then we propose AutoLoss-AR, the first method for searching loss functions for tightening the approximation error of adversarial risk. Extensive experiments are conducted in multiple settings. The results demonstrate the effectiveness of the proposed method: the best-discovered loss functions outperform the handcrafted baseline by 0.9%-2.9% and 0.7%-2.0% on MNIST and CIFAR-10, respectively. Besides, we also verify that the searched losses can be transferred to other settings and explore why they are better than the baseline by visualizing the local loss landscape.

show abstract

Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach

Cited by 61 publications

References 25 publications

Evaluation of Neural Networks Defenses and Attacks using NDCG and Reciprocal Rank Metrics

Evaluation of Neural Networks Defenses and Attacks using NDCG and Reciprocal Rank Metrics

An Integrated Process for Verifying Deep Learning Classifiers Using Dataset Dissimilarity Measures

Tightening the Approximation Error of Adversarial Risk with Auto Loss Function Search

Contact Info

Product

Resources

About