Robustness Analysis of Neural Networks via Efficient Partitioning With Applications in Control Systems

Everett, Michael; Habibi, Golnaz; How, Jonathan P.

doi:10.1109/lcsys.2020.3045323

Cited by 19 publications

(20 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A closed-loop propagator uses the trained NN control policy and dynamics to estimate reachable sets, and a closed-loop partitioner decides how to split the initial state set into pieces. This is the closed-loop extension of the architecture from our prior work [37].…”

Section: F Algorithm For Computing Forward Reachable Setsmentioning

confidence: 99%

“…Recall that [37] introduced an architecture composed of a partitioner and propagator for analyzing NNs in isolation. The partitioner is an algorithm to split the NN input set in an intelligent way (e.g., uniform gridding [36], simulation guidance [20], greedy simulation guidance [37]), and the propagator is an algorithm to estimate bounds on the NN outputs given a NN input set (e.g., IBP [7], Fast-Lin [38], CROWN [10], SDP [13]). The partitioner can query the propagator repeatedly to refine the estimated output set bounds.…”

Section: Tighter Reachable Sets By Partitioning the Initial State Setmentioning

confidence: 99%

“…4. In particular, CL-CROWN and Reach-SDP represent closed-loop propagators (given an initial state set, they compute reachable sets for a trained NN control policy and closed-loop dynamics), and in this section we extend partitioners discussed in [37] to be closedloop partitioners. Altogether, we call the nested architecture a closed-loop analyzer.…”

Section: Tighter Reachable Sets By Partitioning the Initial State Setmentioning

confidence: 99%

Section: A Closed-loop Partitionersmentioning

confidence: 99%

“…There are several reasons to use an iterative partitioner rather than the uniform partitioner presented before, as discussed in [20], [37]. Thus, the closed-loop variant of the greedy simulation-guided partitioner from [37] is described in Algorithm 3. After acquiring N Monte Carlo samples from X 0 (Line 1), those sampled points are simulated forward in time according to the dynamics and control policy (Line 2).…”

Section: A Closed-loop Partitionersmentioning

confidence: 99%

See 4 more Smart Citations

Reachability Analysis of Neural Feedback Loops

Everett¹,

Habibi

How

2021

IEEE Access

Self Cite

View full text Add to dashboard Cite

Neural Networks (NNs) can provide major empirical performance improvements for closedloop systems, but they also introduce challenges in formally analyzing those systems' safety properties. In particular, this work focuses on estimating the forward reachable set of neural feedback loops (closed-loop systems with NN controllers). Recent work provides bounds on these reachable sets, but the computationally tractable approaches yield overly conservative bounds (thus cannot be used to verify useful properties), and the methods that yield tighter bounds are too intensive for online computation. This work bridges the gap by formulating a convex optimization problem for the reachability analysis of closed-loop systems with NN controllers. While the solutions are less tight than previous (semidefinite program-based) methods, they are substantially faster to compute, and some of those computational time savings can be used to refine the bounds through new input set partitioning techniques, which is shown to dramatically reduce the tightness gap. The new framework is developed for systems with uncertainty (e.g., measurement and process noise) and nonlinearities (e.g., polynomial dynamics), and thus is shown to be applicable to real-world systems. To inform the design of an initial state set when only the target state set is known/specified, a novel algorithm for backward reachability analysis is also provided, which computes the set of states that are guaranteed to lead to the target set. The numerical experiments show that our approach (based on linear relaxations and partitioning) gives a 5× reduction in conservatism in 150× less computation time compared to the state-of-the-art. Furthermore, experiments on quadrotor, 270-state, and polynomial systems demonstrate the method's ability to handle uncertainty sources, high dimensionality, and nonlinear dynamics, respectively.

show abstract

Section: F Algorithm For Computing Forward Reachable Setsmentioning

confidence: 99%