Probabilistic Guarantees for Safe Deep Reinforcement Learning

Bacci, Edoardo; Parker, David

doi:10.1007/978-3-030-57628-8_14

Cited by 22 publications

(12 citation statements)

References 46 publications

(35 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…-Google Scholar 14 -Engineering Village (including Compendex) 15 -Web of Science 16 -Science Direct 17 -Scopus 18 -ACM Digital Library 19 -IEEE Xplore 20 Certification of machine learning systems is a relatively new topic. To review the state-of-the-art techniques, we limited the publication date from January 2015 to September 2020 (the month when we started this work) in our searches.…”

Section: Paper Searchmentioning

confidence: 99%

See 1 more Smart Citation

How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review

Tambon,

Laberge,

et al. 2021

Preprint

View full text Add to dashboard Cite

Context: Machine Learning (ML) has been at the heart of many innovations over the past years. However, including it in so-called "safety-critical" systems such as automotive or aeronautic has proven to be very challenging, since the shift in paradigm that ML brings completely changes traditional certification approaches.Objective: This paper aims to elucidate challenges related to the certification of ML-based safety-critical systems, as well as the solutions that are proposed in the literature to tackle them, answering the question "How to Certify Machine Learning Based Safety-critical Systems?".Method: We conduct a Systematic Literature Review (SLR) of research papers published between 2015 to 2020, covering topics related to the certification of ML systems. In total, we identified 217 papers covering topics considered to be the main pillars of ML certification: Robustness, Uncertainty, Explainability, Verification, Safe Reinforcement Learning, and Direct Certification. We analyzed the main trends and problems of each sub-field and provided summaries of the papers extracted.Results: The SLR results highlighted the enthusiasm of the community for this subject, as well as the lack of diversity in term of datasets and type of ML models. It also emphasized the need to further develop connections between academia and industries to deepen the domain study. Finally, it also illustrated This work is supported by the DEEL project CRDPJ 537462-18 funded by the National Science and Engineering Research Council of Canada (NSERC) and the Consortium for Research and Innovation in Aerospace in Québec (CRIAQ), together with its industrial partners Thales Canada inc, Bell Textron Canada Limited, CAE inc and Bombardier inc.

show abstract

Section: Paper Searchmentioning

confidence: 99%

“…An algorithm for measuring the safety of RL agents has been proposed in [17]. A controller is modelled to characterize the actions taken by the agent and the possibilities that result from taking them.…”

Section: Stability Certification Of Rl-based Controllersmentioning

confidence: 99%

How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review

Tambon,

Laberge,

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Our abstraction mechanism is based on the assumption that a trained controller usually adopts the same action for those concrete states that are adjacent [2]. We consider a concrete state s to be a vector of n (n ≥ 1) real numbers.…”

Section: State Discretization and Abstractionmentioning

confidence: 99%

Learning on Abstract Domains: A New Approach for Verifiable Guarantee in Reinforcement Learning

Jin,

Zhang,

et al. 2021

Preprint

View full text Add to dashboard Cite

Formally verifying Deep Reinforcement Learning (DRL) systems is a challenging task due to the dynamic continuity of system behaviors and the black-box feature of embedded neural networks. In this paper, we propose a novel abstraction-based approach to train DRL systems on finite abstract domains instead of concrete system states. It yields neural networks whose input states are finite, making hosting DRL systems directly verifiable using model checking techniques. Our approach is orthogonal to existing DRL algorithms and off-the-shelf model checkers. We implement a resulting prototype training and verification framework and conduct extensive experiments on the state-of-the-art benchmark. The results show that the systems trained in our approach can be verified more efficiently while they retain comparable performance against those that are trained without abstraction.Preprint. Under review.

show abstract

“…Various recent works consider verification or synthesis of RL schemes against reachability specifications [Sun et al, 2019, Könighofer et al, 2020, Bacci and Parker, 2020. None of these approaches, however, support both continuous state-action spaces and probabilistic models, as in this work.…”

Section: Introductionmentioning

confidence: 98%

Certification of Iterative Predictions in Bayesian Neural Networks

Wicker,

Laurenti,

Patane

et al. 2021

Preprint

View full text Add to dashboard Cite

We consider the problem of computing reach-avoid probabilities for iterative predictions made with Bayesian neural network (BNN) models. Specifically, we leverage bound propagation techniques and backward recursion to compute lower bounds for the probability that trajectories of the BNN model reach a given set of states while avoiding a set of unsafe states. We use the lower bounds in the context of control and reinforcement learning to provide safety certification for given control policies, as well as to synthesize control policies that improve the certification bounds. On a set of benchmarks, we demonstrate that our framework can be employed to certify policies over BNNs predictions for problems of more than 10 dimensions, and to effectively synthesize policies that significantly increase the lower bound on the satisfaction probability.

show abstract

Probabilistic Guarantees for Safe Deep Reinforcement Learning

Cited by 22 publications

References 46 publications

How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review

How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review

Learning on Abstract Domains: A New Approach for Verifiable Guarantee in Reinforcement Learning

Certification of Iterative Predictions in Bayesian Neural Networks

Contact Info

Product

Resources

About