Optimized tracking control using reinforcement learning strategy for a class of nonlinear systems

In this paper, for self‐driving forklift, a path‐following framework based on composite disturbance rejection that combines a cascaded active disturbance rejection controller and an online estimator of model parameters is proposed on the basis of a geometry feedforward controller. The model‐based cascaded active disturbance rejection control (ADRC), in particular, is composed of an outer‐loop controller and an inner‐loop controller, with the outer‐loop controller reducing lateral error by manipulating the desired heading direction and the inner‐loop controller achieving heading control by adjusting the steering angle. The unmodeled dynamics of the actual forklift motion are denoted as total disturbances, which can be timely estimated by the extended state observer (ESO). Moreover, to increase the transient response, a parameter self‐learning based geometry feedforward controller is designed in conjunction with the forklift characteristics. The stability analysis is presented and the effectiveness of the composite algorithm is quantitatively evaluated in experiments, demonstrating the superiority over the conventional pure pursuit algorithm and the only use of ADRC.

“…Theorem 2. Consider system (19) and system (22) with controller ( 19)-( 29) and ( 31)-( 33 The closed-loop system can be presented as follows.…”

Section: Stability Analysis Of Adrc Controllermentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Parameter self‐learning feedforward compensation‐based active disturbance rejection for path‐following control of self‐driving forklift trucks

Xie

Song

et al. 2023

“…Subsequently, the event-triggered control function in the event-triggered controller determines whether to update the OBCC strategy based on the values of system states and local neighbor bipartite consensus error. If the event-triggering condition is satisfied, the OBCC strategy is updated using (18), and the event-triggered system states is obtained and transmitted to the ZOH through network. Then, critic-actor NNs in the RL algorithm will solve value function and OBCC policy values to achieve control of MASs and realize bipartite consensus control.…”

Section: Distributed Event-triggered Control and Stability Analysismentioning

confidence: 99%

“…Recently, the RL approach plays an effective role in seeking solutions for HJB equations in optimal control, because it can learn and make decisions based on actor-critic neural networks (NNs). For example, [18] proposes a simplified optimized tracking control strategy based on the RL algorithm. In fact, it utilizes NNs to approximate the solution of the HJB equation, which effectively addresses the issue of extensive computation in solving optimal control and tracking control problems.…”

Section: Introductionmentioning

confidence: 99%

Distributed event‐triggered optimal bipartite consensus control for multiagent systems with input delay via reinforcement learning method

Zhang,

Ma,

Zhang

et al. 2023

In this article, an optimal bipartite consensus control (OBCC) scheme is proposed for heterogeneous multiagent systems (MASs) with input delay by reinforcement learning (RL) algorithm. A directed signed graph is established to construct MASs with competitive and cooperative relationships, and model reduction method is developed to tackle input delay problem. Then, based on the Hamilton–Jacobi–Bellman (HJB) equation, policy iteration method is utilized to design the bipartite consensus controller, which consists of value function and optimal controller. Further, a distributed event‐triggered function is proposed to increase control efficiency, which only requires information from its own agent and neighboring agents. Based on the input‐to‐state stability (ISS) function and Lyapunov function, sufficient conditions for the stability of MASs can be derived. Apart from that, RL algorithm is employed to solve the event‐triggered OBCC problem in MASs, where critic neural networks (NNs) and actor NNs estimate value function and control policy, respectively. Finally, simulation results are given to validate the feasibility and efficiency of the proposed algorithm.

“…Route planning for AGVs is a single‐source shortest route problem, which can be solved by various algorithms, such as Dijkstra's algorithm [1], neural networks [2], rapid random tree (RRT) [3], and probabilistic road map (PRM) [4]. Dijkstra's algorithm is a classical route search algorithm.…”

Section: Introductionmentioning

confidence: 99%

Multi‐AGV route planning in automated warehouse system based on shortest‐time Q‐learning algorithm

Zhang

Chen

Zhao

2023

Route planning for automated guided vehicles (AGVs) is one of the key factors that affects work efficiency of automated storage and retrieval systems (AS/RSes). Route planning plays an important role in the operation of AGVs. Since the characteristic of AS/RSes is chessboard‐like, the environment is more complex than traditional route planning environments because the number of nodes is large, more than one shortest route exists between two nodes, and the routes with the shortest distance may not be the most energy‐saving routes. Although the traditional route planning algorithms such as the classical Q‐learning algorithm can work well in AGV route planning, it also has some limitations. This paper proposes a novel multi‐AGV route planning approach to solving the AGV route planning problem in the chessboard‐like warehouse, which can improve the route planning efficiency greatly. First, by combining adjacency matrix and reward matrix, we propose a low‐dimensional adjacency‐reward matrix for route planning. This algorithm improves the efficiency of classical Q‐learning algorithms and accelerates dynamic route planning significantly. We further improve the algorithm by considering the travel directions to minimize the number of turns in the route and additionally by considering whether the AGV is loaded or not and plan routes accordingly. Finally, we propose a multi‐AGV online collision‐free route planning algorithm based on these considerations for dynamic route planning for multi‐AGVs operating in a large‐scale warehouse. The proposed algorithms are validated with several case studies.