Adaptive Shielding under Uncertainty

Pranger, Stefan; Könighofer, Bettina; Tappler, Martin; Deixelberger, Martin; Jansen, Nils; Bloem, Roderick

doi:10.23919/acc50511.2021.9482889

Cited by 14 publications

(9 citation statements)

References 25 publications

(42 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For future work, we plan to investigate the application of online shielding in other settings, such as decision making in robotics and control. Another interesting extension would be to incorporate quantitative performance measures in the form of rewards and costs into the computation of the online shield, as previously demonstrated in an offline manner [4] and in a hybrid approach [28], where runtime information was used to learn the environment dynamics.…”

Section: Discussionmentioning

confidence: 99%

“…Shields are usually constructed offline by computing a maximally permissive policy that contains all actions that will not violate the safety specification. Several extensions exist [6,37,4,28]. The shielding approach has been shown to be successful in combination with RL [2,21].…”

Section: Related Workmentioning

confidence: 99%

“…Pranger et al [28] proposed adaptive shields to enforce quantitative objectives at run-time. While the computation of their shields is performed offline, the authors deal with the consequences of an incorrect or incomplete model that is used for the computation of the shield.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Online Shielding for Stochastic Systems

Könighofer¹,

Rudolf²,

Palmisano³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper, we propose a method to develop trustworthy reinforcement learning systems. To ensure safety especially during exploration, we automatically synthesize a correct-by-construction runtime enforcer, called a shield, that blocks all actions that are unsafe with respect to a temporal logic specification from the agent. Our main contribution is a new synthesis algorithm for computing the shield online. Existing offline shielding approaches compute exhaustively the safety of all states-action combinations ahead-of-time, resulting in huge offline computation times, large memory consumption, and significant delays at run-time due to the look-ups in a huge database. The intuition behind online shielding is to compute during run-time the set of all states that could be reached in the near future. For each of these states, the safety of all available actions is analysed and used for shielding as soon as one of the considered states is reached. Our proposed method is general and can be applied to a wide range of planning problems with stochastic behavior. For our evaluation, we selected a 2-player version of the classical computer game Snake. The game requires fast decisions and the multiplayer setting induces a large state space, computationally expensive to analyze exhaustively. The safety objective of collision avoidance is easily transferable to a variety of planning tasks.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Online Shielding for Stochastic Systems

Könighofer¹,

Rudolf²,

Palmisano³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In [5] and [6], shields were used to ensure the safety of human-interactive robotics. A shield that adapted to a changing environment was proposed and applied to traffic light controllers to maintain the correct traffic flow [7]. Safe reinforcement learning using shields was proposed in [8] and [9].…”

Section: Introductionmentioning

confidence: 99%

Finite-Horizon Shield for Path Planning Ensuring Safety/Co-Safety Specifications and Security Policies

Kanashima

Ushio

2023

IEEE Access

View full text Add to dashboard Cite

With the development of network technology, security in path planning problems has attracted widespread attention. We consider a path planning problem in which a planner computes a finite path that satisfies a specification. We assume that the specification includes mandatory safety/co-safety specifications. Moreover, we consider a security policy for this path. However, we assume that the information leaked to an intruder is not known beforehand. Then, we propose an enforcement mechanism referred to as a finitehorizon shield. This mechanism modifies the path computed by the planner as small as possible to satisfy the safety/co-safety specifications and security policy under the leaked information. We assume that the safety/co-safety specifications are described by LTL f formulas and the security policy by a hyperLTL f formula. Subsequently, we convert the formulas into quantified formulas and compute the modified path using a satisfiability modulo theories solver. As an example, we consider an opacity problem where there is another path whose leaked information is the same as that of the modified path. By simulations, it confirms that the output of shield depends on the leaked information and the modified path may have additional movements to ensure opacity. We also compare the computation time of the shield with that of a security-aware planning by simulation.

show abstract

“…Another approach is verifying safety on the fly using MPC safety certification [22]. A similar research line is adaptive reinforcement learning [23], [24], where safety is computed for the next k steps and unsafe actions are blocked. There is an intrinsic trade-off when choosing the number k of steps.…”

Section: Introductionmentioning

confidence: 99%

Probabilistic Safety Guarantees for Markov Decision Processes

Wiśniewski

Bujorianu

2023

IEEE Trans. Automat. Contr.

View full text Add to dashboard Cite

This paper aims to incorporate safety specifications into Markov decision processes. Explicitly, we address the minimization problem up to a stopping time with safety constraints. We establish a formalism leaning upon the evolution equation to achieve our goal. We show how to compute the safety function with dynamic programming. In the last part of the paper, we develop several algorithms for safe stochastic optimisation using linear and dynamic programming.

show abstract

Adaptive Shielding under Uncertainty

Cited by 14 publications

References 25 publications

Online Shielding for Stochastic Systems

Online Shielding for Stochastic Systems

Finite-Horizon Shield for Path Planning Ensuring Safety/Co-Safety Specifications and Security Policies

Probabilistic Safety Guarantees for Markov Decision Processes

Contact Info

Product

Resources

About