Approximate Optimal Motion Planning to Avoid Unknown Moving Avoidance Regions

Deptula, Patryk; Chen, Hsi-Yuan; Licitra, Ryan A.; Rosenfeld, Joel A.; Dixon, Warren E.

doi:10.1109/tro.2019.2955321

Cited by 21 publications

(17 citation statements)

References 66 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ADP was successfully extended to address input constrained control problems in Modares et al (2013) and Vamvoudakis et al (2016) . The state-constrained ADP problem was studied in the context of obstacle avoidance in Walters et al (2015) and Deptula et al (2020) , where an additional term that penalizes proximity to obstacles was added to the cost function. Since the added proximity penalty in Walters et al (2015) was finite, the ADP feedback could not guarantee obstacle avoidance, and an auxiliary controller was needed.…”

Section: Introductionmentioning

confidence: 99%

“…Since the added proximity penalty in Walters et al (2015) was finite, the ADP feedback could not guarantee obstacle avoidance, and an auxiliary controller was needed. In Deptula et al (2020) , a barrier-like function was used to ensure unbounded growth of the proximity penalty near the obstacle boundary. While this approach results in avoidance guarantees, it relies on the relatively strong assumption that the value function is continuously differentiable over a compact set that contains the obstacles in spite of penalty-induced discontinuities in the cost function.…”

Section: Introductionmentioning

confidence: 99%

“…In Marvi and Kiumarsi (2021) , the authors develop a safe off-policy RL scheme which trades-off between safety and performance. In Cohen and Belta (2020) , the authors develop a safe RL scheme in which the proximity penalty approach from Deptula et al (2020) is cast into the framework of CBFs. While the control barrier function results in safety guarantees, the existence of a smooth value function, in spite of a nonsmooth cost function, needs to be assumed.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Safe Model-Based Reinforcement Learning for Systems With Parametric Uncertainties

et al. 2021

View full text Add to dashboard Cite

Reinforcement learning has been established over the past decade as an effective tool to find optimal control policies for dynamical systems, with recent focus on approaches that guarantee safety during the learning and/or execution phases. In general, safety guarantees are critical in reinforcement learning when the system is safety-critical and/or task restarts are not practically feasible. In optimal control theory, safety requirements are often expressed in terms of state and/or control constraints. In recent years, reinforcement learning approaches that rely on persistent excitation have been combined with a barrier transformation to learn the optimal control policies under state constraints. To soften the excitation requirements, model-based reinforcement learning methods that rely on exact model knowledge have also been integrated with the barrier transformation framework. The objective of this paper is to develop safe reinforcement learning method for deterministic nonlinear systems, with parametric uncertainties in the model, to learn approximate constrained optimal policies without relying on stringent excitation conditions. To that end, a model-based reinforcement learning technique that utilizes a novel filtered concurrent learning method, along with a barrier transformation, is developed in this paper to realize simultaneous learning of unknown model parameters and approximate optimal state-constrained control policies for safety-critical systems.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Safe Model-Based Reinforcement Learning for Systems With Parametric Uncertainties

et al. 2021

View full text Add to dashboard Cite

show abstract

“…The approaches in [21], [22] rely on a barrier Lyapunov function (BLF) [12] based system transformation to convert a system with rectangular state constraints (i.e., the safe set is a hyper-rectangle) into an equivalent unconstrained system, allowing for the use of unconstrained ADP algorithms to develop control policies that guarantee stability and safety. Other authors have successfully developed ADP methods for particular safety-critical tasks with more complex constraints [25]; however, the developments are task-specific and may not generalize beyond their respective domains. Motivated by the recent success of CBFs in domains such as robotics that require complex safety requirements that cannot be expressed as box constraints on the system states, the authors in [23], [24] incorporate a CBF-based term into the cost function of an optimal control problem to synthesize control policies capable of guaranteeing satisfaction of safety constraints expressed as CBFs.…”

Section: Introductionmentioning

confidence: 99%

“…In contrast, the approach developed in this paper accounts for safe sets defined as the super-level set of a general continuously differentiable function, which can be used to encapsulate a variety of complex safety requirements [9]. In contrast to existing ADP approaches that leverage CBFs [23], [24] (and the closely related motion planning framework [25]), the method developed in this paper does not rely on including a non-differentiable term in the cost function that may compromise the differentiability of the resulting value function. Instead, a novel safeguarding controller is developed that shields the learned policy corresponding to an unconstrained optimal control problem from unsafe actions in a minimally invasive fashion.…”

Section: Introductionmentioning

confidence: 99%

Safe Exploration in Model-based Reinforcement Learning using Control Barrier Functions

Cohen¹,

Belta²

2021

Preprint

View full text Add to dashboard Cite

This paper studies the problem of developing an approximate dynamic programming (ADP) framework for learning online the value function of an infinite-horizon optimal problem while obeying safety constraints expressed as control barrier functions (CBFs). Our approach is facilitated by the development of a novel class of CBFs, termed Lyapunov-like CBFs (LCBFs), that retain the beneficial properties of CBFs for developing minimally-invasive safe control policies while also possessing desirable Lyapunov-like qualities such as positive semi-definiteness. We show how these LCBFs can be used to augment a learning-based control policy so as to guarantee safety and then leverage this approach to develop a safe exploration framework in a model-based reinforcement learning setting. We demonstrate that our developed approach can handle more general safety constraints than state-of-the-art safe ADP methods through a variety of numerical examples.

show abstract

Adaptive dynamic programming for optimal control of discrete‐time nonlinear system with state constraints based on control barrier function

Wang

Rao

et al. 2021

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

Adaptive dynamic programming (ADP) methods have demonstrated their efficiency. However, many of the applications for which ADP offers great potential, are also safety-critical and need to meet safety specifications in the presence of physical constraints. In this article, an optimal controller for solving discrete-time nonlinear systems with state constraints is proposed. By introducing the control barrier function into the utility function, the problem with state constraints is transformed into an unconstrained optimal control problem, addressing state constraints which are difficult to handle by traditional ADP methods. The constructed sequence of value function is shown to be monotonically non-increasing and converges to the optimal value. Besides, this article gives the stability proof of the developed algorithm, as well as the conditions for satisfying the state constraints. To implement and approximate the control barrier function based adaptive dynamic programming algorithm, an actor-critic network structure is built. During the training process, two neural networks are used for approximation separately. The performance of the proposed method is validated by testing it on a simulation example.

show abstract

Approximate Optimal Motion Planning to Avoid Unknown Moving Avoidance Regions

Cited by 21 publications

References 66 publications

Safe Model-Based Reinforcement Learning for Systems With Parametric Uncertainties

Safe Model-Based Reinforcement Learning for Systems With Parametric Uncertainties

Safe Exploration in Model-based Reinforcement Learning using Control Barrier Functions

Adaptive dynamic programming for optimal control of discrete‐time nonlinear system with state constraints based on control barrier function

Contact Info

Product

Resources

About