Adaptive PID controller based on
            <i>Q</i>
            ‐learning algorithm

Shi, Qian; Lam, Hak‐Keung; Xiao, Bo; Tsai, Shun-Hung

doi:10.1049/trit.2018.1007

Cited by 41 publications

(20 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Q-learning algorithm is an offline rule of reinforcement learning (RL) [48], [49]. It approximates and updates the current rule with the optimal action-value (Q * ) based on the action-value (Q).…”

Section: B Deterministic Q-slp Algorithm With a Stable Learning Ratementioning

confidence: 99%

PID Controller Autotuning Design by a Deterministic Q-SLP Algorithm

Pongfai

Zhang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

The proportional integral and derivative (PID) controller is extensively applied in many applications. However, three parameters must be properly adjusted to ensure effective performance of the control system: the proportional gain (K P), integral gain (K I) and derivative gain (K D). Therefore, the aim of this paper is to optimize and improve the stability, convergence and performance in autotuning the PID parameter by using a deterministic Q-SLP algorithm. The proposed method is a combination of the swarm learning process (SLP) algorithm and Q-learning algorithm. The Q-learning algorithm is applied to optimize the weight updating of the SLP algorithm based on the new deterministic rule and closed-loop stabilization of the learning rate. To validate the global optimization of the deterministic rule, it is proven based on the Bellman equation, and the stability of the learning process is proven with respect to the Lyapunov stability theorem. Additionally, to demonstrate the superiority of the performance and convergence in autotuning the PID parameter, simulation results of the proposed method are compared with those based on the central position control (CPC) system using the traditional SLP algorithm, the whale optimization algorithm (WOA) and improved particle swarm optimization (IPSO). The comparison shows that the proposed method can provide results superior to those of the other algorithms with respect to both performance indices and convergence. INDEX TERMS Autotuning gain, central position control system, Q-learning algorithm, PID controller, swarm learning process algorithm, optimal control.

show abstract

Section: B Deterministic Q-slp Algorithm With a Stable Learning Ratementioning

confidence: 99%

PID Controller Autotuning Design by a Deterministic Q-SLP Algorithm

Pongfai

Zhang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Reinforcement Learning is an approach aimed at solving problems such as control systems [18], energy management systems [19][20][21] and is one of the methods of machine learning. The essence of learning influenced this approach because only by communicating with the environment can the control policy produce without understanding the underlying system model.…”

Section: Reinforcement Learningmentioning

confidence: 99%

“…The purpose of the agent is to extract the optimum control strategy to optimize the discounted accumulated rewards, called as expected discounted return G t in the long term, the governing equation of which is given in Ref. [18].…”

Section: Reinforcement Learningmentioning

confidence: 99%

Design and Simulation of Adaptive PID Controller Based on Fuzzy Q-Learning Algorithm for a BLDC Motor

Rr¹,

Nabiyev²,

Ss³

et al. 2020

Preprint

View full text Add to dashboard Cite

Reinforcement learning (RL) is an extensively applied control method for the purpose of designing intelligent control systems to achieve high accuracy as well as better performance. In the present article, the PID controller is considered as the main control strategy for brushless DC (BLDC) motor speed control. For better performance, the fuzzy Q-learning (FQL) method as a reinforcement learning approach is proposed to adjust the PID coefficients. A comparison with the adaptive PID (APID) controller is also performed for the superiority of the proposed method, and the findings demonstrate the reduction of the error of the proposed method and elimination of the overshoot for controlling the motor speed. MATLAB/SIMULINK has been used for modeling, simulation, and control design of the BLDC motor.

show abstract

“…Therefore, it is interesting to establish a hybrid algorithm that combines the intelligent DRL (for example, the aforementioned dueling DQN) algorithm and a traditional PID controller, in order to take advantage of DRL's self-learning capability to tune a PID performance online. Unlike the practice proposed by some literatures [16,30], which uses the reinforcement learning approach to adjust the gains of PID controllers, in this paper, a simpler but more powerful method will be introduced by adding a dueling DQN algorithm directly after a fine-tuned PID controller, as can be seen in Figure 7. There are two special modifications that need to be considered here.…”

Section: Dueling Deep Q-network Architecturementioning

confidence: 99%

A Hybrid End-to-End Control Strategy Combining Dueling Deep Q-network and PID for Transient Boost Control of a Diesel Engine with Variable Geometry Turbocharger and Cooled EGR

et al. 2019

Energies

View full text Add to dashboard Cite

Deep reinforcement learning (DRL), which excels at solving a wide variety of Atari and board games, is an area of machine learning that combines the deep learning approach and reinforcement learning (RL). However, to the authors’ best knowledge, there seem to be few studies that apply the latest DRL algorithms on real-world powertrain control problems. If there are any, the requirement of classical model-free DRL algorithms typically for a large number of random exploration in order to realize good control performance makes it almost impossible to implement directly on a real plant. Unlike most of the other DRL studies, whose control strategies can only be trained in a simulation environment—especially when a control strategy has to be learned from scratch—in this study, a hybrid end-to-end control strategy combining one of the latest DRL approaches—i.e., a dueling deep Q-network and traditional Proportion Integration Differentiation (PID) controller—is built, assuming no fidelity simulation model exists. Taking the boost control of a diesel engine with a variable geometry turbocharger (VGT) and cooled (exhaust gas recirculation) EGR as an example, under the common driving cycle, the integral absolute error (IAE) values with the proposed algorithm are improved by 20.66% and 9.7% respectively for the control performance and generality index, compared with a fine-tuned PID benchmark. In addition, the proposed method can also improve system adaptiveness by adding another redundant control module. This makes it attractive to real plant control problems whose simulation models do not exist, and whose environment may change over time.

show abstract

Adaptive PID controller based on Q ‐learning algorithm

Cited by 41 publications

References 27 publications

PID Controller Autotuning Design by a Deterministic Q-SLP Algorithm

PID Controller Autotuning Design by a Deterministic Q-SLP Algorithm

Design and Simulation of Adaptive PID Controller Based on Fuzzy Q-Learning Algorithm for a BLDC Motor

A Hybrid End-to-End Control Strategy Combining Dueling Deep Q-network and PID for Transient Boost Control of a Diesel Engine with Variable Geometry Turbocharger and Cooled EGR

Contact Info

Product

Resources

About