Optimal Control of Iron-Removal Systems Based on Off-Policy Reinforcement Learning

Chen, Ning; Luo, Shuhan; Dai, Jiayang; Luo, Biao; Gui, Weihua

doi:10.1109/access.2020.3015801

Cited by 8 publications

(2 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While the classical PID controller design is based on the system’s mechanism, the ADP algorithm requires a training model built through monitoring signals. Depending on the iteration rules, the ADP algorithm can be categorized into policy iteration [ 18 , 19 , 20 , 21 , 22 ] and value iteration [ 23 , 24 , 25 , 26 ]. Researchers have explored the application of the ADP control algorithm in exoskeleton controllers, achieving favorable control effects [ 27 , 28 , 29 , 30 , 31 ].…”

Section: Introductionmentioning

confidence: 99%

Tracking Control for a Lower Extremity Exoskeleton Based on Adaptive Dynamic Programing

Su,

Pei,

Tang

2023

Biomimetics

View full text Add to dashboard Cite

The utilization of lower extremity exoskeletons has witnessed a growing presence across diverse domains such as the military, medical treatment, and rehabilitation. This paper introduces a novel design of a lower extremity exoskeleton specifically tailored for individuals engaged in heavy object carrying tasks. The exoskeleton incorporates an impressive 12 degrees of freedom (DOF), with four of them being effectively controlled through hydraulic cylinders. To achieve optimal control of this intricate lower extremity exoskeleton system, the authors propose an adaptive dynamic programming (ADP) algorithm. Several crucial components are established to implement this control scheme. These include the formulation of the state equation for the lower extremity exoskeleton system, which is well-suited for the ADP algorithm. Additionally, a corresponding performance index function based on the tracking error is devised, along with the game algebraic Riccati equation. By employing the value iteration ADP scheme, the lower extremity exoskeleton demonstrates highly effective tracking control. This research not only highlights the potential of the proposed control approach but also showcases its ability to enhance the overall performance and functionality of lower extremity exoskeletons, particularly in scenarios involving heavy object carrying. Overall, this study contributes to the advancement of lower extremity exoskeleton technology and offers valuable insights into the application of ADP algorithms for achieving precise and efficient control in demanding tasks.

show abstract

Section: Introductionmentioning

confidence: 99%

Tracking Control for a Lower Extremity Exoskeleton Based on Adaptive Dynamic Programing

Su,

Pei,

Tang

2023

Biomimetics

View full text Add to dashboard Cite

show abstract

“…With the rapid development of optimal control theory [1] , optimal control methods have been widely used in industry process and aerospace fields [2][3][4] . Optimal control theory can be traced back to the 1950s, then Siebenthal [5] solved the stirred reactor optimization control problem based on the Pontryagin maximum principle.…”

Section: Introductionmentioning

confidence: 99%

Design of LQR Tracking Controller Combined With Orthogonal Collocation State Planning for Process Optimal Control

Liu

Teng

et al. 2020

IEEE Access

View full text Add to dashboard Cite

As one of the most important optimization methods for process optimal control, orthogonal collocation method has been widely used. However, this kind of optimization method is generally an open-loop optimization frame, which makes the model disturbances greatly affect the control performance and quality. In order to improve the optimal control performance of dynamic systems with disturbances, this work proposed an orthogonal collocation optimization-based linear quadratic regulator (OC-LQR) control method for process optimal control problems. Firstly, the orthogonal collocation method is derived in detail to transform the optimal control problem so that optimal state curves can be accordingly calculated. Then, an improved LQR controller design is proposed by using the obtained optimal state curves so as to construct the feedback control frame. On this basis, a state planning-based LQR tracking control is established with closed-loop control characteristics to tackle model disturbance problem. Meanwhile, the detailed Simulink model of the proposed control method is constructed for simulation execution. Finally, the proposed control structure is tested on a classical process optimal control problem with model disturbance (white noise) test and Gaussian mixture noise test to verify the performance of the proposed method. Simulation studies show that the control performance of the proposed method is excellent, where the mean absolute error of state curve tracking averagely reduces by 34.73% and the minimal performance index improves by 64.44% when compared with the classical orthogonal optimal control method. Simulation results demonstrate the effectiveness of the proposed planning-based tracking control framework.

show abstract

A survey and comparative evaluation of actor‐critic methods in process control

Dutta

Upreti

2022

Can J Chem Eng

View full text Add to dashboard Cite

Actor-critic (AC) methods have emerged as an important class of reinforcement learning (RL) paradigm that enables model-free control by acting on a process and learning from the consequence. To that end, these methods utilize artificial neural networks, which are synergized for action evaluation and optimal action prediction. This feature is highly desirable for process control, especially when the knowledge about a process is limited or when it is susceptible to uncertainties. In this work, we summarize important concepts of AC methods and survey their process control applications. This treatment is followed by a comparative evaluation of the set-point tracking and robustness of controllers based on five prominent AC methods, namely, DDPG, TD3, SAC, PPO, and TRPO, in five case studies of varying process nonlinearity. The training demands and control performances indicate the superiority of DDPG and TD3 methods, which rely on off-policy, deterministic search for optimal action policies. Overall, the knowledge base and results of this work are expected to serve practitioners in their efforts toward further development of autonomous process control strategies.

show abstract

Optimal Control of Iron-Removal Systems Based on Off-Policy Reinforcement Learning

Abstract: I confirm that I hold the necessary rights to share the selected file with the intended recipients.Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000.

Cited by 8 publications

References 34 publications

Tracking Control for a Lower Extremity Exoskeleton Based on Adaptive Dynamic Programing

Tracking Control for a Lower Extremity Exoskeleton Based on Adaptive Dynamic Programing

Design of LQR Tracking Controller Combined With Orthogonal Collocation State Planning for Process Optimal Control

A survey and comparative evaluation of actor‐critic methods in process control

Contact Info

Product

Resources

About