Neural Network Verification in Control

Abstract

Learning-based methods could provide solutions to many of the long-standing challenges in control. However, the neural networks (NNs) commonly used in modern learning approaches present substantial challenges for analyzing the resulting control systems' safety properties. Fortunately, a new body of literature could provide tractable methods for analysis and verification of these high dimensional, highly nonlinear representations. This tutorial first introduces and unifies recent techniques (many of which originated in the computer vision and machine learning communities) for verifying robustness properties of NNs. The techniques are then extended to provide formal guarantees of neural feedback loops (e.g., closed-loop system with NN control policy). The provided tools are shown to enable closed-loop reachability analysis and robust deep reinforcement learning.

Training a Resilient Q-network against Observational Interference

“…Adversarial Perturbation: An intensifying challenge against deep neural network based systems is adversarial perturbation for making incorrect decisions. Many gradientbased noise-generating methods (Goodfellow, Shlens, and Szegedy 2015;Huang et al 2017;Everett 2021) have been conducted for misclassification and mislead an agent's output action. As an example of using DRL model playing Atari games, an adversarial attacker (Lin et al 2017;Yang et al 2020) could jam in a timely and barely detectable noise to maximize the prediction loss of a Q-network and cause massively degraded performance.…”

Parametric Chordal Sparsity for SDP-based Neural Network Verification

“…Another way of approaching safety verification is by reformulating it as a reachability problem, see e.g., [20], [21], [22], [23], [24], [25] which has been of particular interest for the safety verification of closed-loop dynamical systems [26]. While heuristics for reachability problems over dynamical systems exist, these methods are typically computationally expensive.…”