A novel optimal control design for unknown nonlinear systems based on adaptive dynamic programming and nonlinear model predictive control

Hu, Wei; Zhang, Guoshan; Zheng, Yuqing

doi:10.1002/asjc.2552

Cited by 8 publications

(10 citation statements)

References 37 publications

(78 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The initial value of neural network remarkably affects the control performance and training convergence speed [20,21]. At the initial stage of agent searching the data points, if the Q-value estimated by the initial neural network is far from the ideal Q-value, the RL algorithm needs more samples and longer training time to converge.…”

Section: Neural Network Initializationmentioning

confidence: 99%

B-Spline-Based Curve Fitting to Cam Pitch Curve Using Reinforcement Learning

Lin¹,

Chen²,

Jiang³

et al. 2023

Intelligent Automation &Amp; Soft Computing

View full text Add to dashboard Cite

Directly applying the B-spline interpolation function to process plate cams in a computer numerical control (CNC) system may produce verbose tool-path codes and unsmooth trajectories. This paper is devoted to addressing the problem of B-spline fitting for cam pitch curves. Considering that the B-spline curve needs to meet the motion law of the follower to approximate the pitch curve, we use the radial error to quantify the effects of the fitting B-spline curve and the pitch curve. The problem thus boils down to solving a difficult global optimization problem to find the numbers and positions of the control points or data points of the B-spline curve such that the cumulative radial error between the fitting curve and the original curve is minimized, and this problem is attempted in this paper with a double deep Q-network (DDQN) reinforcement learning (RL) algorithm with data points traceability. Specifically, the RL environment, actions set and current states set are designed to facilitate the search of the data points, along with the design of the reward function and the initialization of the neural network. The experimental results show that when the angle division value of the actions set is fixed, the proposed algorithm can maximize the number of data points of the B-spline curve, and accurately place these data points to the right positions, with the minimum average of radial errors. Our work establishes the theoretical foundation for studying spline fitting using the RL method.

show abstract

Section: Neural Network Initializationmentioning

confidence: 99%

B-Spline-Based Curve Fitting to Cam Pitch Curve Using Reinforcement Learning

Lin¹,

Chen²,

Jiang³

et al. 2023

Intelligent Automation &Amp; Soft Computing

View full text Add to dashboard Cite

show abstract

“…According to earlier studies [11,12], the optimal control problem can be solved by minimizing the infinite horizon quadratic cost function. However, in this paper, because of security constraints, the traditional quadratic cost function cannot achieve our goal.…”

Section: Time-triggered Safe Control Schemementioning

confidence: 99%

“…Different from the TTC scheme, in the event-triggered safe control scheme, the update of the controller does not use the state of the current instant but uses the sampled state of the event-triggering instants to update. Therefore, a zero-order holder is adopted between event-triggering instants; according to (11), we can get the event-triggered optimal safety control law as follows:…”

Section: Event-triggered Safe Control Schemementioning

confidence: 99%

“…According to (11) and ( 13) forms of control law and the Assumption 2, then we can derive the following triggering condition.…”

Section: Event-triggered Safe Control Schemementioning

confidence: 99%

“…In the design of optimal safety controller, if the method QP-based is abandoned, we will face the solution of the common optimal control problem. As we all know, solving the Hamilton-Jacobi-Bellman (HJB) equation can get the optimal solution of the system [11,12]. However, for many nonlinear systems, it is almost impossible to get the analytical solution because of its own partial differential properties [13].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Adaptive critic learning for event‐triggered safe control of nonlinear safety‐critical systems

Qin

Zhu

Wang

et al. 2023

Asian Journal of Control

View full text Add to dashboard Cite

In this paper, an event‐triggered safe control method based on adaptive critic learning (ACL) is proposed for a class of nonlinear safety‐critical systems. First, a safe cost function is constructed by adding a control barrier function (CBF) to the traditional quadratic cost function; the optimization problem with safety constraints that is difficult to deal with by classical ACL methods is solved. Subsequently, the event‐triggered scheme is introduced to reduce the amount of computation. Further, combining the properties of CBF with the ACL‐based event‐triggering mechanism, the event‐triggered safe Hamilton–Jacobi–Bellman (HJB) equation is derived, and a single critic neural network (NN) framework is constructed to approximate the solution of the event‐triggered safe HJB equation. In addition, the concurrent learning method is applied to the NN learning process, so that the persistence of excitation (PE) condition is not required. The weight approximation error of the NN and the states of the system are proven to be uniformly ultimately bounded (UUB) in the safe set with the Lyapunov theory. Finally, the availability of the presented method can be validated through the simulation.

show abstract

Event‐triggered‐based integral reinforcement learning output feedback optimal control for partially unknown constrained‐input nonlinear systems

Zou

Zhang

2023

Asian Journal of Control

Self Cite

View full text Add to dashboard Cite

In this paper, an adaptive output feedback event‐triggered optimal control algorithm is proposed for partially unknown constrained‐input continuous‐time nonlinear systems. First, a neural network observer is constructed to estimate unmeasurable state. Next, an event‐triggered condition is established, and only when the event‐triggered condition is violated will the event be triggered and the state be sampled. Then, an event‐triggered‐based synchronous integral reinforcement learning (ET‐SIRL) control algorithm with critic‐actor neural networks (NNs) architecture is proposed to solve the event‐triggered Hamilton–Jacobi–Bellman equation under the established event‐triggered condition. The critic and actor NNs are used to approximate cost function and optimal event‐triggered optimal control law, respectively. Meanwhile, the event‐triggered‐based closed‐loop system state and all the neural network weight estimation errors are uniformly ultimately bounded proved by Lyapunov stability theory, and there is no Zeno behavior. Finally, two numerical examples are presented to show the effectiveness of the proposed ET‐SIRL control algorithm.

show abstract

A novel optimal control design for unknown nonlinear systems based on adaptive dynamic programming and nonlinear model predictive control

Cited by 8 publications

References 37 publications

B-Spline-Based Curve Fitting to Cam Pitch Curve Using Reinforcement Learning

B-Spline-Based Curve Fitting to Cam Pitch Curve Using Reinforcement Learning

Adaptive critic learning for event‐triggered safe control of nonlinear safety‐critical systems

Event‐triggered‐based integral reinforcement learning output feedback optimal control for partially unknown constrained‐input nonlinear systems

Contact Info

Product

Resources

About