Neural Network Verification using Polynomial Optimisation

Newton, Matthew; Papachristodoulou, Antonis

doi:10.1109/cdc45484.2021.9683286

Cited by 6 publications

(5 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These constraints can be of any form, however common choices are box, slope and sector constraints due to their ability to be expressed in a semi-definite programming framework. In our framework, we are able to choose any set of polynomial constraints to represent the NN, as shown in previous work [19], [20]. These polynomial constraints can be represented as a semi-algebraic set, which we express with notation…”

Section: A Abstracting the Neural Network As Constraintsmentioning

confidence: 99%

See 1 more Smart Citation

Stability of Non-linear Neural Feedback Loops using Sum of Squares

Newton¹,

Papachristodoulou²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Neural network controllers have the potential to improve the performance of feedback systems compared to traditional controllers, due to their ability to act as general function approximators. However, quantifying their safety and robustness properties has proven challenging due to the nonlinearities of the activation functions inside the neural network.A key robustness indicator is certifying the stability properties of the feedback system and providing a region of attraction, which has been addressed in previous literature. However, these works only address linear systems or require one to abstract the plant non-linearities and bound them using slope and sector constraints. In this paper we use a Sum of Squares programming framework to compute the stability of non-linear systems with neural network controllers directly. Within this framework, we can propose higher order candidate Lyapunov functions with richer structures that are able to better capture the dynamics of the non-linear system and the nonlinearities in the neural network. We are also able to analyse these systems in continuous time, whereas other methods rely on discretising the system. These higher order Lyapunov functions are used in conjunction with higher order multipliers on the inequality and equality constraints that bound the neural network inputoutput properties. The volume of the region of attraction computed is increased compared to other methods, allowing for better safety guarantees on the stability of the system. We are also able to easily analyse non-linear polynomial systems, which is not possible to do with other methods. We are also able to conduct robustness analysis on the parameter uncertainty. We show the benefits of our method using numerical examples.

show abstract

Section: A Abstracting the Neural Network As Constraintsmentioning

confidence: 99%

“…where α is determined by the IBP bounds. Alternatively we can use two overlapping sectors which can provide better constraints on the activation function when the IBP bounds are large [19].…”

Section: A Abstracting the Neural Network As Constraintsmentioning

confidence: 99%

Stability of Non-linear Neural Feedback Loops using Sum of Squares

Newton¹,

Papachristodoulou²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…For the sigmoid and tanh activation functions it is possible to create two overlapping sector constraints using the IBP values. [15] provides more details about these constraints.…”

Section: Neural Network Constraintsmentioning

confidence: 99%

Reachability Analysis of Neural Feedback Loops using Sparse Polynomial Optimisation

Newton

Papachristodoulou

2022

2022 IEEE 61st Conference on Decision and Control (CDC)

Self Cite

View full text Add to dashboard Cite

Neural networks have seen a recent increased use in control feedback systems. However, providing robustness guarantees on these feedback systems has proven challenging and to combat these issues, there is a significant amount of current research. One of the biggest shortcoming of neural networks is how sensitive they are to adversarial inputs. Given that feedback systems are usually subject to external perturbations, this issue surrounding neural networks must be overcome before they can be used in safety-critical applications. One method to tackle this problem is to compute outer-approximations of the reachable sets, through bounding the activation functions in the neural network controller. Our approach is to use these bounds in a sparse polynomial optimisation framework in conjunction with the Positivstellensatz. The sparsity property is able to exploit the natural cascading structure of the neural network to allow for tractable solve times. The Positivstellensatz is able to provide more accurate bounds over similar methods by asserting the emptiness of a semi-algebraic set. We show through examples that our method can provide tighter bounds over similar methods, with reasonable computational time. Our approach is also able to deal with non-linear polynomial dynamics due to the polynomial optimisation framework we use, while other methods need to use alternative work-around solutions to incorporate the non-linearities.

show abstract

“…To alleviate the computational cost, a memory-efficient first-order algorithm was introduced in (Dathathri et al 2020). With the same aim, layerwise SDP relaxations were used in (Batten et al 2021;Newton and Papachristodoulou 2021) by exploiting the cascaded NN structures based on chordal graph decomposition (Zheng, Fantuzzi, and Papachristodoulou 2021). To the best of our knowledge, the LayerSDP method (Batten et al 2021) achieves the tightest relaxations by combining SDP relaxation with triangle relaxation (Ehlers 2017).…”

Section: Introductionmentioning

confidence: 99%

Iteratively Enhanced Semidefinite Relaxations for Efficient Neural Network Verification

Lan

Zheng

Lomuscio

2023

AAAI

View full text Add to dashboard Cite

We propose an enhanced semidefinite program (SDP) relaxation to enable the tight and efficient verification of neural networks (NNs). The tightness improvement is achieved by introducing a nonlinear constraint to existing SDP relaxations previously proposed for NN verification. The efficiency of the proposal stems from the iterative nature of the proposed algorithm in that it solves the resulting non-convex SDP by recursively solving auxiliary convex layer-based SDP problems. We show formally that the solution generated by our algorithm is tighter than state-of-the-art SDP-based solutions for the problem. We also show that the solution sequence converges to the optimal solution of the non-convex enhanced SDP relaxation. The experimental results on standard benchmarks in the area show that our algorithm achieves the state-of-the-art performance whilst maintaining an acceptable computational cost.

show abstract

Neural Network Verification using Polynomial Optimisation

Cited by 6 publications

References 18 publications

Stability of Non-linear Neural Feedback Loops using Sum of Squares

Stability of Non-linear Neural Feedback Loops using Sum of Squares

Reachability Analysis of Neural Feedback Loops using Sparse Polynomial Optimisation

Iteratively Enhanced Semidefinite Relaxations for Efficient Neural Network Verification

Contact Info

Product

Resources

About