Verifying Learning-Based Robotic Navigation Systems

Amir, Guy; Corsi, Davide; Yerushalmi, Raz; Marzari, Luca; Harel, David; Farinelli, Alessandro; Katz, Guy

doi:10.1007/978-3-031-30823-9_31

Cited by 9 publications

(2 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An example of this process appears in Appendix B of our extended paper [7]. To date, a plethora of verification approaches have been proposed for general, feed-forward DNNs [3,31,41,46,61,99], as well as DRL-based agents that operate within reactive environments [5,9,15,22,28].…”

Section: Dnn and Drl Verificationmentioning

confidence: 99%

Verifying Generalization in Deep Learning

Amir

Osher

Tom

et al. 2023

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Deep neural networks (DNNs) are the workhorses of deep learning, which constitutes the state of the art in numerous application domains. However, DNN-based decision rules are notoriously prone to poor generalization, i.e., may prove inadequate on inputs not encountered during training. This limitation poses a significant obstacle to employing deep learning for mission-critical tasks, and also in real-world environments that exhibit high variability. We propose a novel, verification-driven methodology for identifying DNN-based decision rules that generalize well to new input domains. Our approach quantifies generalization to an input domain by the extent to which decisions reached by independently trained DNNs are in agreement for inputs in this domain. We show how, by harnessing the power of DNN verification, our approach can be efficiently and effectively realized. We evaluate our verification-based approach on three deep reinforcement learning (DRL) benchmarks, including a system for Internet congestion control. Our results establish the usefulness of our approach. More broadly, our work puts forth a novel objective for formal verification, with the potential for mitigating the risks associated with deploying DNN-based systems in the wild.

show abstract

Section: Dnn and Drl Verificationmentioning

confidence: 99%

Verifying Generalization in Deep Learning

Amir

Osher

Tom

et al. 2023

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…In particular, DNNs are often considered "black-box" systems, meaning their internal representation is not fully transparent. A crucial DNNs weakness is the vulnerability to adversarial attacks (Szegedy et al 2013;Amir et al 2023), wherein small, imperceptible modifications to input data can lead to wrong and potentially catastrophic decisions when deployed.…”

Section: Introductionmentioning

confidence: 99%

Enumerating Safe Regions in Deep Neural Networks with Provable Probabilistic Guarantees

Marzari,

Corsi,

Marchesini

et al. 2024

AAAI

View full text Add to dashboard Cite

Identifying safe areas is a key point to guarantee trust for systems that are based on Deep Neural Networks (DNNs). To this end, we introduce the AllDNN-Verification problem: given a safety property and a DNN, enumerate the set of all the regions of the property input domain which are safe, i.e., where the property does hold. Due to the #P-hardness of the problem, we propose an efficient approximation method called ε-ProVe. Our approach exploits a controllable underestimation of the output reachable sets obtained via statistical prediction of tolerance limits, and can provide a tight —with provable probabilistic guarantees— lower estimate of the safe areas. Our empirical evaluation on different standard benchmarks shows the scalability and effectiveness of our method, offering valuable insights for this new type of verification of DNNs.

show abstract

Verifying Learning-Based Robotic Navigation Systems

Amir

Corsi

Yerushalmi

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Deep reinforcement learning (DRL) has become a dominant deep-learning paradigm for tasks where complex policies are learned within reactive systems. Unfortunately, these policies are known to be susceptible to bugs. Despite significant progress in DNN verification, there has been little work demonstrating the use of modern verification tools on real-world, DRL-controlled systems. In this case study, we attempt to begin bridging this gap, and focus on the important task of mapless robotic navigation — a classic robotics problem, in which a robot, usually controlled by a DRL agent, needs to efficiently and safely navigate through an unknown arena towards a target. We demonstrate how modern verification engines can be used for effective model selection, i.e., selecting the best available policy for the robot in question from a pool of candidate policies. Specifically, we use verification to detect and rule out policies that may demonstrate suboptimal behavior, such as collisions and infinite loops. We also apply verification to identify models with overly conservative behavior, thus allowing users to choose superior policies, which might be better at finding shorter paths to a target. To validate our work, we conducted extensive experiments on an actual robot, and confirmed that the suboptimal policies detected by our method were indeed flawed. We also demonstrate the superiority of our verification-driven approach over state-of-the-art, gradient attacks. Our work is the first to establish the usefulness of DNN verification in identifying and filtering out suboptimal DRL policies in real-world robots, and we believe that the methods presented here are applicable to a wide range of systems that incorporate deep-learning-based agents.

show abstract

Verifying Learning-Based Robotic Navigation Systems

Cited by 9 publications

References 43 publications

Verifying Generalization in Deep Learning

Verifying Generalization in Deep Learning

Enumerating Safe Regions in Deep Neural Networks with Provable Probabilistic Guarantees

Verifying Learning-Based Robotic Navigation Systems

Contact Info

Product

Resources

About