GoTube: Scalable Statistical Verification of Continuous-Depth Models

Gruenbacher, Sophie; Lechner, Mathias; Hasani, Ramin; Rus, Daniela; Henzinger, Thomas A.; Smolka, Scott A.; Grosu, Radu

doi:10.1609/aaai.v36i6.20631

Cited by 8 publications

(4 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some notable examples are Sherlock (Dutta, Chen, and Sankaranarayanan 2019) and ReachNN/ReachNN * (Huang et al 2019;Fan et al 2020) which use polynomial approximations to overapproximate the reachable set over a given time horizon, NNV (Tran et al 2020) which is based on abstract interpretation, LRT-NG (Gruenbacher et al 2020) which overapproximates the reachable set as sequence of hyperspheres, or Verisig (Ivanov et al 2019) which reduces the problem to reachability analysis in hybrid systems. Furthermore, Go-Tube (Gruenbacher et al 2021) constructs the reachable set of a deterministic continuous-time system with statistical guarantees about the constructed set overapproximating the true reachable states.…”

Section: Reachability For Deterministic Control Problemsmentioning

confidence: 99%

Stability Verification in Stochastic Control Systems via Neural Network Supermartingales

Lechner

Žikelić

Chatterjee

et al. 2022

AAAI

Self Cite

View full text Add to dashboard Cite

We consider the problem of formally verifying almost-sure (a.s.) asymptotic stability in discrete-time nonlinear stochastic control systems. While verifying stability in deterministic control systems is extensively studied in the literature, verifying stability in stochastic control systems is an open problem. The few existing works on this topic either consider only specialized forms of stochasticity or make restrictive assumptions on the system, rendering them inapplicable to learning algorithms with neural network policies. In this work, we present an approach for general nonlinear stochastic control problems with two novel aspects: (a) instead of classical stochastic extensions of Lyapunov functions, we use ranking supermartingales (RSMs) to certify a.s. asymptotic stability, and (b) we present a method for learning neural network RSMs. We prove that our approach guarantees a.s. asymptotic stability of the system and provides the first method to obtain bounds on the stabilization time, which stochastic Lyapunov functions do not. Finally, we validate our approach experimentally on a set of nonlinear stochastic reinforcement learning environments with neural network policies.

show abstract

Section: Reachability For Deterministic Control Problemsmentioning

confidence: 99%

Stability Verification in Stochastic Control Systems via Neural Network Supermartingales

Lechner

Žikelić

Chatterjee

et al. 2022

AAAI

Self Cite

View full text Add to dashboard Cite

show abstract

“…Similar to RNNs, neural ODEs are also deep learning models with "memory", which makes them suitable to learn time-series data, but are also applicable to other tasks such as continuous normalizing flows (CNF) and image classification [11,61]. However, existing work is limited to a stochastic reachability approach [27,28], reachability approaches using star and zonotope reachability methods for a general class of neural ODEs (GNODE) with continuous and discrete time layers [52], and GAINS [89], which leverages ODE-solver information to discretize the models using a computation graph that represent all possible trajectories from a given input to accelerate their bound propagation method. However, one of the main challenges is to find a framework that is able to verify several of these models successfully.…”

Section: Related Workmentioning

confidence: 99%

NNV 2.0: The Neural Network Verification Tool

Lopez

Choi

Tran

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

This manuscript presents the updated version of the Neural Network Verification (NNV) tool. NNV is a formal verification software tool for deep learning models and cyber-physical systems with neural network components. NNV was first introduced as a verification framework for feedforward and convolutional neural networks, as well as for neural network control systems. Since then, numerous works have made significant improvements in the verification of new deep learning models, as well as tackling some of the scalability issues that may arise when verifying complex models. In this new version of NNV, we introduce verification support for multiple deep learning models, including neural ordinary differential equations, semantic segmentation networks and recurrent neural networks, as well as a collection of reachability methods that aim to reduce the computation cost of reachability analysis of complex neural networks. We have also added direct support for standard input verification formats in the community such as VNNLIB (verification properties), and ONNX (neural networks) formats. We present a collection of experiments in which NNV verifies safety and robustness properties of feedforward, convolutional, semantic segmentation and recurrent neural networks, as well as neural ordinary differential equations and neural network control systems. Furthermore, we demonstrate the capabilities of NNV against a commercially available product in a collection of benchmarks from control systems, semantic segmentation, image classification, and time-series data.

show abstract

“…Neural networks have been shown to generalize well to unseen test data on a large range of tasks, despite achieving zero loss on training data (Zhang et al, 2017(Zhang et al, , 2021. But this performance is useless if neural networks cannot be used in practice due to safety and security issues (Gruenbacher et al, 2022;Grunbacher et al, 2021;Lechner et al, 2020Xiao et al, 2021Xiao et al, , 2022. A fundamental question in the security of neural networks is how much information is leaked via this training procedure, that is, can adversaries with access to trained models, or predictions from a model, infer what data was used to train the model?…”

Section: Introductionmentioning

confidence: 99%

Dataset Distillation Fixes Dataset Reconstruction Attacks

Noel¹,

Hasani²,

Lechner³

et al. 2023

Preprint

View full text Add to dashboard Cite

Modern deep learning requires large volumes of data, which could contain sensitive or private information which cannot be leaked. Recent work has shown for homogeneous neural networks a large portion of this training data could be reconstructed with only access to the trained network parameters. While the attack was shown to work empirically, there exists little formal understanding of its effectiveness regime, and ways to defend against it. In this work, we first build a stronger version of the dataset reconstruction attack and show how it can provably recover their entire training set in the infinite width regime. We then empirically study the characteristics of this attack on two-layer networks and reveal that its success heavily depends on deviations from the frozen infinite-width Neural Tangent Kernel limit. More importantly, we formally show for the first time that dataset reconstruction attacks are a variation of dataset distillation. This key theoretical result on the unification of dataset reconstruction and distillation not only sheds more light on the characteristics of the attack but enables us to design defense mechanisms against them via distillation algorithms.

show abstract

GoTube: Scalable Statistical Verification of Continuous-Depth Models

Cited by 8 publications

References 24 publications

Stability Verification in Stochastic Control Systems via Neural Network Supermartingales

Stability Verification in Stochastic Control Systems via Neural Network Supermartingales

NNV 2.0: The Neural Network Verification Tool

Dataset Distillation Fixes Dataset Reconstruction Attacks

Contact Info

Product

Resources

About