Revisiting PGD Attacks for Stability Analysis of Large-Scale Nonlinear Systems and Perception-Based Control

Abstract

Many existing region-of-attraction (ROA) analysis tools find difficulty in addressing feedback systems with large-scale neural network (NN) policies and/or high-dimensional sensing modalities such as cameras. In this paper, we tailor the projected gradient descent (PGD) attack method developed in the adversarial learning community as a general-purpose ROA analysis tool for large-scale nonlinear systems and end-to-end perception-based control. We show that the ROA analysis can be approximated as a constrained maximization problem whose goal is to find the worst-case initial condition which shifts the terminal state the most. Then we present two PGD-based iterative methods which can be used to solve the resultant constrained maximization problem. Our analysis is not based on Lyapunov theory, and hence requires minimum information of the problem structures. In the model-based setting, we show that the PGD updates can be efficiently performed using backpropagation. In the model-free setting (which is more relevant to ROA analysis of perception-based control), we propose a finite-difference PGD estimate which is general and only requires a blackbox simulator for generating the trajectories of the closed-loop system given any initial state. We demonstrate the scalability and generality of our analysis tool on several numerical examples with large-scale NN policies and high-dimensional image observations. We believe that our proposed analysis serves as a meaningful initial step toward further understanding of closed-loop stability of large-scale nonlinear systems and perception-based control.

Revisiting PGD Attacks for Stability Analysis of High-Dimensional Nonlinear Systems and Perception-Based Control

“…We apply the PGD update rule (11) with the finite difference gradient estimator (12) to perform ROA analysis. More discussion on the alternative PGD update rule ( 9) is given in our arXiv report [25]. Finally, we will also justify our approach via a comparison with a Monte Carlo sampling baseline.…”

“…For other values of p, (10) can also be applied. Due to the page limit, we only discuss the update rules for these other cases in our arXiv report [25]. Another subtle issue is how to choose T. Notice that T cannot be too large.…”

“…We present our ROA analysis results for perception-based control of nonlinear cartpole systems (with single and double links). For the full-state feedback case, comparison to an existing quadratic-constraint approach can be found in our arXiv report [25], however it is not applicable to this setting. The perception-based feedback control loop is visualized in Figure 2.…”

Performance-Robustness Tradeoffs in Adversarially Robust Linear-Quadratic Control

“…Al Makdah et al (2020) proves the existence of performance-robustness tradeoffs in control but does not characterize them quantitatively. The extension of adversarial robustness results in machine learning to various control problems is an active area of research (Lee et al, 2021;Havens et al, 2022).…”