Challenging SMT solvers to verify neural networks

Pulina, Luca; Tacchella, Armando

doi:10.3233/aic-2012-0525

Cited by 101 publications

(64 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In worst-case safety verification, we assume that the input uncertainty is bounded and we verify a safety property for all possible perturbations within the uncertainty set. This approach has been pursued extensively in several works using various tools, such as mixed-integer linear programming [6]- [8], robust optimization and duality theory [9], [10], Satisfiability Modulo Theory (SMT) [11], dynamical systems [12], [13], Robust Control [14], Abstract Interpretation [15] and many others [16], [17].…”

Section: Introductionmentioning

confidence: 99%

Probabilistic Verification and Reachability Analysis of Neural Networks via Semidefinite Programming

Fazlyab

Morari

Pappas

2019

2019 IEEE 58th Conference on Decision and Control (CDC)

View full text Add to dashboard Cite

Quantifying the robustness of neural networks or verifying their safety properties against input uncertainties or adversarial attacks have become an important research area in learning-enabled systems. Most results concentrate around the worst-case scenario where the input of the neural network is perturbed within a norm-bounded uncertainty set. In this paper, we consider a probabilistic setting in which the uncertainty is random with known first two moments. In this context, we discuss two relevant problems: (i) probabilistic safety verification, in which the goal is to find an upper bound on the probability of violating a safety specification; and (ii) confidence ellipsoid estimation, in which given a confidence ellipsoid for the input of the neural network, our goal is to compute a confidence ellipsoid for the output. Due to the presence of nonlinear activation functions, these two problems are very difficult to solve exactly. To simplify the analysis, our main idea is to abstract the nonlinear activation functions by a combination of affine and quadratic constraints they impose on their input-output pairs. We then show that the safety of the abstracted network, which is sufficient for the safety of the original network, can be analyzed using semidefinite programming. We illustrate the performance of our approach with numerical experiments.

show abstract

Section: Introductionmentioning

confidence: 99%

Probabilistic Verification and Reachability Analysis of Neural Networks via Semidefinite Programming

Fazlyab

Morari

Pappas

2019

2019 IEEE 58th Conference on Decision and Control (CDC)

View full text Add to dashboard Cite

show abstract

“…Unfortunately, this verification problem is NP-complete [13], making it theoretically difficult. It is also difficult in practice, with modern solvers scaling only to very small examples [18,19]. Because problems involving only linear constraints are fairly easy to solve, many solvers handle the ReLU constraints by transforming the input query into a sequence of pure linear sub-problems, such that the original query is satisfiable if and only if at least one of the sub-problems is satisfiable.…”

Section: Verifying Properties Of Neural Networkmentioning

confidence: 99%

Towards Proving the Adversarial Robustness of Deep Neural Networks

Katz

Barrett

Dill

et al. 2017

Electron. Proc. Theor. Comput. Sci.

View full text Add to dashboard Cite

Autonomous vehicles are highly complex systems, required to function reliably in a wide variety of situations. Manually crafting software controllers for these vehicles is difficult, but there has been some success in using deep neural networks generated using machine-learning. However, deep neural networks are opaque to human engineers, rendering their correctness very difficult to prove manually; and existing automated techniques, which were not designed to operate on neural networks, fail to scale to large systems. This paper focuses on proving the adversarial robustness of deep neural networks, i.e. proving that small perturbations to a correctly-classified input to the network cannot cause it to be misclassified. We describe some of our recent and ongoing work on verifying the adversarial robustness of networks, and discuss some of the open questions we have encountered and how they might be addressed.

show abstract

“…In contrast to our work, these methods do not actually establish verified properties on the input-output behavior of ANNs. Formal methods-based approaches for verifying ANNs include abstraction-refinement based approaches [20], bounded model checking for neural network for control problems [23] and neural network verification using SMT solvers or other specialized solvers [21,13,11]. Instead we rely on solving MIP problems and parallelization of branch-and-bound algorithms.…”

Section: Introductionmentioning

confidence: 99%

Maximum Resilience of Artificial Neural Networks

Cheng

Nührenberg

Rueß

2017

Automated Technology for Verification and Analysis

197

178

View full text Add to dashboard Cite

The deployment of Artificial Neural Networks (ANNs) in safety-critical applications poses a number of new verification and certification challenges. In particular, for ANN-enabled self-driving vehicles it is important to establish properties about the resilience of ANNs to noisy or even maliciously manipulated sensory input. We are addressing these challenges by defining resilience properties of ANN-based classifiers as the maximum amount of input or sensor perturbation which is still tolerated. This problem of computing maximum perturbation bounds for ANNs is then reduced to solving mixed integer optimization problems (MIP). A number of MIP encoding heuristics are developed for drastically reducing MIP-solver runtimes, and using parallelization of MIP-solvers results in an almost linear speed-up in the number (up to a certain limit) of computing cores in our experiments. We demonstrate the effectiveness and scalability of our approach by means of computing maximum resilience bounds for a number of ANN benchmark sets ranging from typical image recognition scenarios to the autonomous maneuvering of robots.

show abstract

Challenging SMT solvers to verify neural networks

Cited by 101 publications

References 28 publications

Probabilistic Verification and Reachability Analysis of Neural Networks via Semidefinite Programming

Probabilistic Verification and Reachability Analysis of Neural Networks via Semidefinite Programming

Towards Proving the Adversarial Robustness of Deep Neural Networks

Maximum Resilience of Artificial Neural Networks

Contact Info

Product

Resources

About