Adversarial robustness assessment: Why in evaluation both L0 and L∞ attacks are necessary

Kotyan, Shashank; Vargas, Danilo Vasconcellos

doi:10.1371/journal.pone.0265723

Cited by 14 publications

(7 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…37 In contrast to the present study, which assumed no learning (i.e., no plasticity) in the network during a task, previous pioneering experiments of robotic embodiments of a living neuronal network exploited Hebbian plasticity within networks to optimize sensory-motor coupling for a given task. [22][23][24][25][26][27][28] These contradictory strategies in embodiment experiments have confirmed that both the homeostatic property and Hebbian learning play substantial roles in task solving in the brain. 29,30 A synergetic effect should be con ment experiments and in the theory of brain…”

Section: Theoretical Background On Morphological Computationmentioning

confidence: 80%

“…Recently, there has been a resurgence of interest in RC as a compelling biological model of neural networks, driven by its potential for scalability through physical implementations. Additionally, problems in deep learning such as adversarial attacks and robustness/adaptiveness issues have also contributed to an increase in interest in alternative paradigms [17,18,19,20,21,22].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning

Zhang

Vargas

2023

IEEE Access

Self Cite

View full text Add to dashboard Cite

Reservoir computing (RC), first applied to temporal signal processing, is a recurrent neural network in which neurons are randomly connected. Once initialized, the connection strengths remain unchanged. Such a simple structure turns RC into a non-linear dynamical system that maps lowdimensional inputs into a high-dimensional space. The model's rich dynamics, linear separability, and memory capacity then enable a simple linear readout to generate adequate responses for various applications. RC spans areas far beyond machine learning, since it has been shown that the complex dynamics can be realized in various physical hardware implementations and biological devices. This yields greater flexibility and shorter computation time. Moreover, the neuronal responses triggered by the model's dynamics shed light on understanding brain mechanisms that also exploit similar dynamical processes. While the literature on RC is vast and fragmented, here we conduct a unified review of RC's recent developments from machine learning to physics, biology, and neuroscience. We first review the early RC models, and then survey the state-of-the-art models and their applications. We further introduce studies on modeling the brain's mechanisms by RC. Finally, we offer new perspectives on RC development, including reservoir design, coding frameworks unification, physical RC implementations, and interaction between RC, cognitive neuroscience and evolution.

show abstract

Section: Theoretical Background On Morphological Computationmentioning

confidence: 80%

Section: Introductionmentioning

confidence: 99%

A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning

Zhang

Vargas

2023

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…In addition, the choice of perturbation level by the L0-norm or L∞-norm metric does not depend on the image size, which is convenient for comparison. In the experiments, the formation of attacks is proposed to be implemented on the basis of the search algorithm of the covariance matrix adaptation evolution strategy (CMA-ES) using the L∞ metric [33].…”

Section: Results Of Machine Learning and Discussionmentioning

confidence: 99%

Neural network based image classifier resilient to destructive perturbation influences – architecture and training method

Москаленко

Moskalenko

2022

reks

View full text Add to dashboard Cite

Modern methods of image recognition are sensitive to various types of disturbances, which actualize the development of resilient intelligent algorithms for safety-critical applications. The current article develops a model and method of training a classifier that exhibits characteristics of resilience to adversarial attacks, fault injection, and concept drift. The proposed model has a hierarchical structure of prototypes and hyperspherical boundaries of classes formed in the space of high-level features. Class boundaries are optimized during training and provide perturbation absorption and graceful degradation. The proposed learning method involves the use of a combined loss function, which allows the use of both labeled and unlabeled data, implements the compression of the feature representation to a discrete form and ensures the compactness of the distribution of classes and the maximization of the buffer zone between classes. The main component of the loss function is the value of the normalized modification of Shannon's information measure, averaged over the alphabet of the classes, expressed as a function of accuracy characteristics. Simultaneously, accuracy characteristics are calculated on the basis of smoothed versions of the distribution of statistical hypothesis testing results. It is experimentally confirmed that the proposed approach provides a certain level of disturbance absorption, graceful degradation and recovery. During testing of the proposed algorithm on the Cifar10 data set, it was established that the integral metric of resilience to tensor damage by inversion of one randomly selected bit is about 0.95 if the share of damaged tensors does not exceed 30%. Also, during testing of the proposed algorithm, it was established that an adversarial attack with a disturbance that does not exceed the L∞-norm threshold equal to 3 provides resilience that exceeds the value of 0.95 according to the integral metric. Additionally, the integral metric of resilience during adaptation to the appearance of two new classes is 0.959. The integral metric of resilience to the real drift of concepts between the two classes is 0.973. The ability to adapt to the appearance of new classes or the concept drift has been confirmed 8 times faster than learning from scratch.

show abstract

“…The experimental results showed that the method can improve the success rate of the attack while maintaining the advantage of having a low degree of perturbation. In proposing a model-independent dual-quality assessment for adversarial machine learning, Vargas et al [ 37 , 38 ] developed the Covariance Matrix Adaptation Evolution Strategy for a novel black-box attack, verifying the effectiveness of the adaptive strategy in improving the OPA performance. After that, Su et al in [ 39 ] further showed the promises of evolutionary computation.…”

Section: Related Workmentioning

confidence: 99%

“…Our method, as one of the optimization methods of the OPA, was also compared with other optimization schemes such as Jing Adaptive Differential Evolution (JADE) [ 35 ], Particle Swarm-based Optimization (PSO) [ 36 ], and Covariance Matrix Adaptation Evolution Strategy (CMA-ES) [ 37 , 38 ] (described in detail in Section 2 ). These methods also aimed to implement the adversarial example attack by modifying the image with only very few pixels.…”

Section: Experiments and Analysismentioning

confidence: 99%

Image Adversarial Example Generation Method Based on Adaptive Parameter Adjustable Differential Evolution

Lin

Peng

Tan

et al. 2023

Entropy

View full text Add to dashboard Cite

Adversarial example generation techniques for neural network models have exploded in recent years. In the adversarial attack scheme for image recognition models, it is challenging to achieve a high attack success rate with very few pixel modifications. To address this issue, this paper proposes an adversarial example generation method based on adaptive parameter adjustable differential evolution. The method realizes the dynamic adjustment of the algorithm performance by adjusting the control parameters and operation strategies of the adaptive differential evolution algorithm, while searching for the optimal perturbation. Finally, the method generates adversarial examples with a high success rate, modifying just a very few pixels. The attack effectiveness of the method is confirmed in CIFAR10 and MNIST datasets. The experimental results show that our method has a greater attack success rate than the One Pixel Attack based on the conventional differential evolution. In addition, it requires significantly less perturbation to be successful compared to global or local perturbation attacks, and is more resistant to perception and detection.

show abstract

Adversarial robustness assessment: Why in evaluation both L0 and L∞ attacks are necessary

Cited by 14 publications

References 25 publications

A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning

A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning

Neural network based image classifier resilient to destructive perturbation influences – architecture and training method

Image Adversarial Example Generation Method Based on Adaptive Parameter Adjustable Differential Evolution

Contact Info

Product

Resources

About