Reinforcement Learning-based Nonlinear Disturbance Observer for UAV with Parametric Uncertainty and Unmodeled Dynamics

Using Deep Reinforcement Learning for Dynamic Gain Adjustment of a Disturbance Observer

“…In this context, it is possible to adjust the gain to reduce the estimation error or noise component. Note that the measured state x m (t) is used instead of the actual state x(t) when implementing the DOB; thus, the noise component will be included in the error dynamics (9) as in (7). Consider a tracking control problem such that x(t) → x * (t), where x * (t) is the reference signal.…”

“…In [6], receding-horizon optimization-based gain tuning of nonlinear DOB was proposed to balance *This work was supported by a Korea Institute for Advancement of Technology (KIAT) grant funded by the Korea Government (MOTIE) (P0020535, The Competency Development Program for Industry Specialist) 1 Kyunghwan Choi and Hyochan Lee are with the School of Mechanical Engineering, Gwangju Institute of Science and Technology, Gwangju 61005, Republic of Korea khchoi@gist.ac.kr; hyochanlee@gm.gist.ac.kr 2 Wooyong Kim is with the Department of Biomedical & Robotics Engineering, Incheon National University, Incheon 22012, Republic of Korea wooyongkim@inu.ac.kr disturbance estimation accuracy and noise suppression. DRL was utilized to optimize the gains of a nonlinear DOB and an active disturbance rejection controller in [7] and [8], respectively, to improve the disturbance rejection performance. A novel paradigm was presented in [9], where a DOB was designed by recurrent neural networks (RNNs) and DRL was utilized to optimize the RNNs for a target environment.…”

“…Conventional DOB design methods [2]- [4] have endeavored to reduce the sensor noise effect and derive robust stability conditions. DRL-based methods [7]- [9] have focused on improving disturbance rejection performance. However, no prior report has investigated a method to simultaneously resolve this trade-off by improving disturbance estimation accuracy and reducing noise sensitivity.…”