The mathematics of adversarial attacks in AI -- Why deep learning is unstable despite the existence of stable neural networks

Abstract

The unprecedented success of deep learning (DL) makes it unchallenged when it comes to classification problems. However, it is well established that the current DL methodology produces universally unstable neural networks (NNs). The instability problem has caused an enormous research effort -with a vast literature on so-called adversarial attacks -yet there has been no solution to the problem. Our paper addresses why there has been no solution to the problem, as we prove the following mathematical paradox: any training procedure based on training neural networks for classification problems with a fixed architecture will yield neural networks that are either inaccurate or unstable (if accurate) -despite the provable existence of both accurate and stable neural networks for the same classification problems. The key is that the stable and accurate neural networks must have variable dimensions depending on the input, in particular, variable dimensions is a necessary condition for stability.Our result points towards the paradox that accurate and stable neural networks exist, however, modern algorithms do not compute them. This yields the question: if the existence of neural networks with desirable properties can be proven, can one also find algorithms that compute them? There are cases in mathematics where provable existence implies computability, but will this be the case for neural networks? The contrary is true, as we demonstrate how neural networks can provably exist as approximate minimisers to standard optimisation problems with standard cost functions, however, no randomised algorithm can compute them with probability better than 1/2.

Adversarial Parameter Attack on Deep Neural Networks

“…Since F Θ is a continuous function in Θ, if Θ a is an adversarial parameter for Θ then there exists a small sphere S a with Θ a as center such that all parameters in S a are also adversarial parameters for Θ. These results imply that adversarial parameters are inevitable in certain sense, similar to adversarial samples [6,3,27].…”

“…Existence of adversarial samples were usually demonstrated with numerical experiments, and mathematical theories were desired. In [6], it was proved that for DNNs with a fixed architecture, there exist uncountable classification functions and distributions of samples such that adversarial samples always exist for any successfully trained DNN with the given architecture and the sample distribution. In the stealth attack [34,35], it was proved that there exist attached DNNs which give a desired label for a sample outside of the validation set by modifying the DNN.…”

“…. This is a variant of R 4 (F, D x ) in (6) with p = 2. The following theorem shows that adversarial parameters exist for this robustness measure.…”

“…Existence of adversary samples makes the DNN vulnerable in safety-critical applications and many effective methods were proposed to develop more robust DNNs against adversarial attacks [20,1,5,42]. However, it was shown that adversaries samples are inevitable for current DNN models in certain sense [6,3,27].…”

The extended Smale's 9th problem -- On computational barriers and paradoxes in estimation, regularisation, computer-assisted proofs and learning

“…The techniques developed in this paper to prove Theorem 3.3 can be used and extended to produce computability and complexity results in other fields. In particular, they form the basis of some of the developments in [7] and [34] on the limitations of AI and Smale's 18th problem.…”