The Marabou Framework for Verification and Analysis of Deep Neural Networks

Katz, Guy; Huang, Derek A.; Ibeling, Duligur; Julian, Kyle D.; Lazarus, Christopher; Lim, Rachel; Shah, Parth; Thakoor, Shantanu; Wu, Haoze; Zeljić, Aleksandar; Dill, David L.; Kochenderfer, Mykel J.; Barrett, Clark

doi:10.1007/978-3-030-25540-4_26

Cited by 358 publications

(318 citation statements)

References 15 publications

Supporting

Mentioning

318

Contrasting

Order By: Relevance

“…We evaluate NNV in comparison to Reluplex [ 22 ], Marabou [ 23 ], and ReluVal [ 49 ], by considering the verification of safety property and of the ACAS Xu neural networks [ 21 ] for all 45 networks. 2 All the experiments were done using 4 cores for computation.…”

Section: Discussionmentioning

confidence: 99%

“…For the "open-loop" verification problem (verification of DNNs), many efficient techniques have been proposed, such as SMT-based methods [22,23,30], mixed-integer linear programming methods [14,24,28], setbased methods [4,17,32,33,48,50,53,57], and optimization methods [51,58]. For the "closed-loop" verification problem (NCCS verification), we note that the Verisig approach [20] is efficient for NNCS with nonlinear plants and with Sigmoid and Tanh activation functions.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

NNV: The Neural Network Verification Tool for Deep Neural Networks and Learning-Enabled Cyber-Physical Systems

Tran

Yang

Lopez

et al. 2020

Computer Aided Verification

158

141

View full text Add to dashboard Cite

This paper presents the Neural Network Verification (NNV) software tool, a set-based verification framework for deep neural networks (DNNs) and learning-enabled cyber-physical systems (CPS). The crux of NNV is a collection of reachability algorithms that make use of a variety of set representations, such as polyhedra, star sets, zonotopes, and abstract-domain representations. NNV supports both exact (sound and complete) and over-approximate (sound) reachability algorithms for verifying safety and robustness properties of feed-forward neural networks (FFNNs) with various activation functions. For learning-enabled CPS, such as closed-loop control systems incorporating neural networks, NNV provides exact and over-approximate reachability analysis schemes for linear plant models and FFNN controllers with piecewise-linear activation functions, such as ReLUs. For similar neural network control systems (NNCS) that instead have nonlinear plant models, NNV supports overapproximate analysis by combining the star set analysis used for FFNN controllers with zonotope-based analysis for nonlinear plant dynamics building on CORA. We evaluate NNV using two real-world case studies: the first is safety verification of ACAS Xu networks, and the second deals with the safety verification of a deep learning-based adaptive cruise control system.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

NNV: The Neural Network Verification Tool for Deep Neural Networks and Learning-Enabled Cyber-Physical Systems

Tran

Yang

Lopez

et al. 2020

Computer Aided Verification

158

141

View full text Add to dashboard Cite

show abstract

“…Early work [59] used abstract interpretation [16] to verify small-sized neural networks. Recent work [25,35,66] used SMT [3] techniques and considered new abstract domains.…”

Section: Related Workmentioning

confidence: 99%

DeepSearch: a simple and effective blackbox attack for deep neural networks

Zhang

Chowdhury

Christakis

2020

Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Softw

View full text Add to dashboard Cite

Although deep neural networks have been very successful in imageclassification tasks, they are prone to adversarial attacks. To generate adversarial inputs, there has emerged a wide variety of techniques, such as black-and whitebox attacks for neural networks. In this paper, we present DeepSearch, a novel fuzzing-based, queryefficient, blackbox attack for image classifiers. Despite its simplicity, DeepSearch is shown to be more effective in finding adversarial inputs than state-of-the-art blackbox approaches. DeepSearch is additionally able to generate the most subtle adversarial inputs in comparison to these approaches. CCS CONCEPTS • Computing methodologies → Neural networks; • Software and its engineering → Software testing and debugging.

show abstract

“…research areas in the MP space include hybrid program synthesis techniques [6,19], automatic test creation [1,16], automatic bug repair [4], human-in-the-loop code recommendation systems [10,18,33], and guaranteeing approximate software solutions using formal methods [5,14,15]. This list is non-exhaustive, and is intended to provide an abbreviated snapshot of active research areas in MP.…”

Section: Related Workmentioning

confidence: 99%

Learned garbage collection

Cen

Marcus

Mao

et al. 2020

Proceedings of the 4th ACM SIGPLAN International Workshop on Machine Learning and Programming Languages

View full text Add to dashboard Cite

Several programming languages use garbage collectors (GCs) to automatically manage memory for the programmer. Such collectors must decide when to look for unreachable objects to free, which can have a large performance impact on some applications. In this preliminary work, we propose a design for a learned garbage collector that autonomously learns over time when to perform collections. By using reinforcement learning, our design can incorporate user-defined reward functions, allowing an autonomous garbage collector to learn to optimize the exact metric the user desires (e.g., request latency or queries per second). We conduct an initial experimental study on a prototype, demonstrating that an approach based on tabular Q learning may be promising.

show abstract

The Marabou Framework for Verification and Analysis of Deep Neural Networks

Cited by 358 publications

References 15 publications

NNV: The Neural Network Verification Tool for Deep Neural Networks and Learning-Enabled Cyber-Physical Systems

NNV: The Neural Network Verification Tool for Deep Neural Networks and Learning-Enabled Cyber-Physical Systems

DeepSearch: a simple and effective blackbox attack for deep neural networks

Learned garbage collection

Contact Info

Product

Resources

About