A Proposal of Adaptive PID Controller Based on Reinforcement Learning

Wang, Xuesong; Cheng, Yuhu; Sun, Wei

doi:10.1016/s1006-1266(07)60009-1

Cited by 64 publications

(20 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Early approaches in adaptive control discuss the advantages of using the tracking error as an update law for adaptive controllers based on adaptive inverse dynamics control [30], [31]. More recent work combines the idea of adaptive control with machine learning to obtain controllers whose parameters evolve with the dynamics of the process [32], [33]. In this paper, we use an adaptive PD controller with fixed proportional gain and variable derivative gain to compute the torque applied to each actuated joint.…”

Section: Adaptive Pd Controllermentioning

confidence: 99%

Reinforcement Learning Meets Hybrid Zero Dynamics: A Case Study for RABBIT

Castillo

Weng

Hereid

et al. 2019

2019 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

The design of feedback controllers for bipedal robots is challenging due to the hybrid nature of its dynamics and the complexity imposed by high-dimensional bipedal models. In this paper, we present a novel approach for the design of feedback controllers using Reinforcement Learning (RL) and Hybrid Zero Dynamics (HZD). Existing RL approaches for bipedal walking are inefficient as they do not consider the underlying physics, often requires substantial training, and the resulting controller may not be applicable to real robots. HZD is a powerful tool for bipedal control with local stability guarantees of the walking limit cycles. In this paper, we propose a non traditional RL structure that embeds the HZD framework into the policy learning. More specifically, we propose to use RL to find a control policy that maps from the robot's reduced order states to a set of parameters that define the desired trajectories for the robot's joints through the virtual constraints. Then, these trajectories are tracked using an adaptive PD controller. The method results in a stable and robust control policy that is able to track variable speed within a continuous interval. Robustness of the policy is evaluated by applying external forces to the torso of the robot. The proposed RL framework is implemented and demonstrated in OpenAI Gym with the MuJoCo physics engine based on the well-known RABBIT robot model.

show abstract

Section: Adaptive Pd Controllermentioning

confidence: 99%

Reinforcement Learning Meets Hybrid Zero Dynamics: A Case Study for RABBIT

Castillo

Weng

Hereid

et al. 2019

2019 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

show abstract

“…RL agents are generally instructed by instant numerical rewards to approach their optimal behavior . To take some instances, they have been employed for control design in several works …”

Section: Control Designmentioning

confidence: 99%

“…40 To take some instances, they have been employed for control design in several works. [41][42][43] The learning algorithms can be fallen into three categories including supervised, unsupervised, and reinforcement learning (RL). Despite artificial neural network and GA, which have been studied in some of the previous works in active structural control area, RL has not been paid attention to a satisfactory level for the purpose of parameters learning despite having distinctive features.…”

Section: Control Designmentioning

confidence: 99%

Online control of an active seismic system via reinforcement learning

Khalatbarisoltani

Soleymani

Khodadadi

2018

Struct Control Health Monit

View full text Add to dashboard Cite

Summary Tuning the seismic control systems in order to achieve optimal performance is a challenging area due to the system and disturbance uncertainties. Although, model uncertainties, process time delay, and actuator dynamics can be considered as typical uncertainties, the main source of uncertainty in a seismic control system comes from the aleatory nature of earthquake disturbances. In this case, tuning of the control system based on a given seismic record may not necessarily result in optimal performance for other earthquakes. In this paper, a methodological approach is proposed for online control of active structural control systems considering seismic uncertainties. For this purpose, the concept of reinforcement learning is utilized for online tuning of an active mass drive system. The controller comprises a gain‐scheduling fuzzy proportional derivative controller whose gains are tuned via an online reinforcement learning algorithm. Moreover, in order to tackle the time delays, a dynamic state predictor is utilized in conjunction with the proposed controller. To evaluate the performance of proposed controller, according to an assumed site hazard, thousand ground motion records are generated and clustered based on their spectral features using a fuzzy clustering approach. Finally, the controller is implemented in a laboratory‐scale structure, and its performance is examined in the presence of the cluster centers and some real seismic records simulated on a shake table. The test results reveal successful performance of the proposed controller in tackling a wide range of seismic disturbances in the presence of time delay.

show abstract

“…Reinforcement Learning [10] permits to adapt on-line the controller parameters to the driver and to the vehicle. As a result, the control layer is able to adapt information provided by the driver (maximum desired velocity, acceleration, suspension behavior...) to each vehicle capabilities at any moment of its lifetime cycle.…”

Section: Adaptabilitymentioning

confidence: 99%