Adaptive dynamic programming for optimal control of discrete‐time nonlinear system with state constraints based on control barrier function

Abstract

Adaptive dynamic programming (ADP) methods have demonstrated their efficiency. However, many of the applications for which ADP offers great potential, are also safety-critical and need to meet safety specifications in the presence of physical constraints. In this article, an optimal controller for solving discrete-time nonlinear systems with state constraints is proposed. By introducing the control barrier function into the utility function, the problem with state constraints is transformed into an unconstrained optimal control problem, addressing state constraints which are difficult to handle by traditional ADP methods. The constructed sequence of value function is shown to be monotonically non-increasing and converges to the optimal value. Besides, this article gives the stability proof of the developed algorithm, as well as the conditions for satisfying the state constraints. To implement and approximate the control barrier function based adaptive dynamic programming algorithm, an actor-critic network structure is built. During the training process, two neural networks are used for approximation separately. The performance of the proposed method is validated by testing it on a simulation example.

Robust Trajectory Tracking Control for Continuous-Time Nonlinear Systems with State Constraints and Uncertain Disturbances

“…The application of the CBF further solves the constraint problem of the system [ 36 ]. In a predefined security set, the CBF candidate is always positive and tends to infinity at the defined set boundary.…”

“…Inspired by references [ 29 , 36 ], is combined with the nominal augmented system ( 13 ), and the modified value function is where = , , the maximum eigenvalue of R can be expressed by , both and are weighted symmetric positive definite matrices of augmented systems, and is a discount coefficient.…”

“…In reference [ 35 ], the application of the CBF was introduced, and the verification method and the characteristics of implementation safety in the context of a safety-critical control system are summarized. The discrete-time state constraints problem was described in reference [ 36 ], and the HJB equation with the CBF was solved by using the approximate properties of the neural network.…”

Learning‐based model predictive control under value iteration with finite approximation errors

“…In recent years, the impressive advances in the field of reinforcement learning 8,9 (RL) and approximate dynamic programming [10][11][12] (also called adaptive dynamic programming, ADP) have captured the attention of the control community. The intrinsic relevance of these learning techniques to MPC, coupled with their powerful online optimization capabilities, has sparked efforts to combine them with MPC to assist controller design while improving performance.…”

Finite-time convergence of perturbed nonlinear systems using adaptive barrier-function nonsingular sliding mode control with experimental validation

“…Generally, at the aim of performance improvement and robustness achievement of the nonlinear systems, the concept of the nonlinear control can be considered as an important tool (Chen et al, 2008; Fei et al, 2021; Xu et al, 2021). Various control techniques including fuzzy control (Yang et al, 2021), neural learning (Zhang et al, 2020a), feedback linearization (Moreno–Valenzuela et al, 2020), event-trigger consensus control (Yao et al, 2020), optimal control (Chignoli and Wensing, 2020), reinforcement learning (Zhao et al, 2020), backstepping control (Liu et al, 2020a), model-predictive control (Liang et al, 2020), and output-feedback control (Redaud et al, 2021) have been used for the control of the nonlinear systems.…”