Policy compression for aircraft collision avoidance systems

Julian, Kyle D.; Lopez, Jessica; Brush, Jeffrey S.; Owen, Michael P.; Kochenderfer, Mykel J.

doi:10.1109/dasc.2016.7778091

Cited by 209 publications

(182 citation statements)

References 14 publications

Supporting

Mentioning

166

Contrasting

Unclassified

Order By: Relevance

“…Training. We trained our verification policy on 12 different robustness properties of a neural network used in the ACAS Xu collision avoidance system [24]. However, since even verifying even a single benchmark can take a very long time, our implementation uses two tactics to reduce training time.…”

Section: Methodsmentioning

confidence: 99%

“…In recent years, deep neural networks (DNNs) have gained enormous popularity for a wide spectrum of applications, ranging from image recognition [21,28] and malware detection [57,58] to machine translation [55]. Due to their surprising effectiveness in practice, deep learning has also found numerous applications in safety-critical systems, including self-driving cars [3,5], unmanned aerial systems [24], and medical diagnosis [9].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Optimization and abstraction: a synergistic approach for analyzing neural network robustness

Anderson

Pailoor

Dillig

et al. 2019

Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation

View full text Add to dashboard Cite

In recent years, the notion of local robustness (or robustness for short) has emerged as a desirable property of deep neural networks. Intuitively, robustness means that small perturbations to an input do not cause the network to perform misclassifications. In this paper, we present a novel algorithm for verifying robustness properties of neural networks. Our method synergistically combines gradient-based optimization methods for counterexample search with abstraction-based proof search to obtain a sound and (δ -)complete decision procedure. Our method also employs a data-driven approach to learn a verification policy that guides abstract interpretation during proof search. We have implemented the proposed approach in a tool called Charon and experimentally evaluated it on hundreds of benchmarks. Our experiments show that the proposed approach significantly outperforms three state-of-the-art tools, namely AI 2 , Reluplex, and Reluval.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Optimization and abstraction: a synergistic approach for analyzing neural network robustness

Anderson

Pailoor

Dillig

et al. 2019

Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation

View full text Add to dashboard Cite

show abstract

“…After that, the DNN-based autonomous vehicle will be able to "execute recognition, prediction, and planning" driving tasks in diverse conditions [10]. Nowadays, CNNs are the most widely adopted deep-learning model for fully autonomous vehicles [5,6,7,8]. NVIDIA introduced an AI supercomputer for autonomy [33].…”

Section: Autonomous Carsmentioning

confidence: 99%

“…The next generation of collision avoidance systems for unmanned aircraft (ACAS Xu) adopted DNN to compress large score table [5]. Julian et al [94] explored the performance of ACAS Xu by measuring a set of safety and performance metrics.…”

Section: Ca4: Assuring Safety Property Of Nn-based Sccpssmentioning

confidence: 99%

Testing and verification of neural-network-based safety-critical control software: A systematic literature review

Zhang

2020

Information and Software Technology

View full text Add to dashboard Cite

Context: Neural Network (NN) algorithms have been successfully adopted in a number of Safety-Critical Cyber-Physical Systems (SCCPSs). Testing and Verification (T&V) of NN-based control software in safety-critical domains are gaining interest and attention from both software engineering and safety engineering researchers and practitioners. Objective: With the increase in studies on the T&V of NN-based control software in safety-critical domains, it is important to systematically review the state-of-the-art T&V methodologies, to classify approaches and tools that are invented, and to identify challenges and gaps for future studies. Method: By searching the six most relevant digital libraries, we retrieved 950 papers on the T&V of NN-based Safety-Critical Control Software (SCCS). Then we filtered the papers based on the predefined inclusion and exclusion criteria and applied snowballing to identify new relevant papers. Results: To reach our result, we selected 83 primary papers published between 2001 and 2018, applied the thematic analysis approach for analyzing the data extracted from the selected papers, presented the classification of approaches, and identified challenges. Conclusion: The approaches were categorized into five high-order themes: assuring robustness of NNs, assuring safety properties of NN-based control software, improving the failure resilience of NNs, measuring and ensuring test completeness, and improving the interpretability of NNs. From the industry perspective, improving the interpretability of NNs is a crucial need in safetycritical applications. We also investigated nine safety integrity properties within four major safety lifecycle phases to investigate the achievement level of T&V goals in IEC 61508-3. Results show that correctness, completeness, freedom from intrinsic faults, and fault tolerance have drawn most attention from the research community. However, little effort has been invested in achieving repeatability; no reviewed study focused on precisely defined testing configuration or on defense against common cause failure.

show abstract

“…For example, much of the technology behind autonomous and driver-less vehicle development is "powered" by machine learning [5,15,4]. DNNs have also been used in airborne collision avoidance systems for unmanned aircraft (ACAS Xu) [24]. However, in designing and deploying these algorithms in critical cyber-physical systems, the presence of an active adversary is often ignored.…”

Section: Introductionmentioning

confidence: 99%

Semantic Adversarial Deep Learning

Dreossi

Jha

Seshia

2018

Computer Aided Verification

View full text Add to dashboard Cite

Abstract.Fueled by massive amounts of data, models produced by machine-learning (ML) algorithms, especially deep neural networks, are being used in diverse domains where trustworthiness is a concern, including automotive systems, finance, health care, natural language processing, and malware detection. Of particular concern is the use of ML algorithms in cyber-physical systems (CPS), such as self-driving cars and aviation, where an adversary can cause serious consequences.However, existing approaches to generating adversarial examples and devising robust ML algorithms mostly ignore the semantics and context of the overall system containing the ML component. For example, in an autonomous vehicle using deep learning for perception, not every adversarial example for the neural network might lead to a harmful consequence. Moreover, one may want to prioritize the search for adversarial examples towards those that significantly modify the desired semantics of the overall system. Along the same lines, existing algorithms for constructing robust ML algorithms ignore the specification of the overall system. In this paper, we argue that the semantics and specification of the overall system has a crucial role to play in this line of research. We present preliminary research results that support this claim.

show abstract

Policy compression for aircraft collision avoidance systems

Cited by 209 publications

References 14 publications

Optimization and abstraction: a synergistic approach for analyzing neural network robustness

Optimization and abstraction: a synergistic approach for analyzing neural network robustness

Testing and verification of neural-network-based safety-critical control software: A systematic literature review

Semantic Adversarial Deep Learning

Contact Info

Product

Resources

About