Reinforcement learning control of a single‐link flexible robotic manipulator

Ouyang, Yuncheng; He, Wei; Li, Xiajing

doi:10.1049/iet-cta.2016.1540

Cited by 62 publications

(35 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Saeed et al (2019) and Badfar and Abdollahi (2019) modeled the dynamics of rigid-flexible manipulators using the AMM technique. Other recent works on single-link FLMs using AMM include, but are not limited to, the studies done by Ouyang et al (2017), Reddy and Jacob (2017), Meng et al (2018), and Zhang et al (2019a). Furthermore, the AMM was used to model two-link FLMs by Lochan and Roy (2018), , and Lochan et al (2020).…”

Section: Assumed Mode Methods (Amm)mentioning

confidence: 99%

“…They claimed that the proposed adaptive neural network controller had better performance than the PD controller. Ouyang et al (2017) proposed a reinforcement learning control to suppress the vibration of a single-link flexible manipulator by using two radial basis function neural networks: actor neural network to design proper control input and critic neural network to approximate the cost function of the system. Sun et al (2016) used adaptive neural networks for control design using full-state feedback and output feedback separately to suppress the vibration of single-link flexible manipulator and highlighted better control performance than the PD control strategy.…”

Section: Model-free Control Techniquesmentioning

confidence: 99%

See 1 more Smart Citation

Review on Modeling and Control of Flexible Link Manipulators

Subedi¹,

Tyapin²,

Hovland³

2020

MIC

View full text Add to dashboard Cite

This paper presents a review of dynamic modeling techniques and various control schemes to control flexible link manipulators (FLMs) that were studied in recent literature. The advantages and complexities associated with the FLMs are discussed briefly. A survey of the reported studies is carried out based on the method used for modeling link flexibility and obtaining equations of motion of the FLMs. The control techniques are reviewed by classifying them into two main categories: model-based and model-free control schemes. The merits and limitations of different modeling and control methods are highlighted.

show abstract

Section: Assumed Mode Methods (Amm)mentioning

confidence: 99%

Section: Model-free Control Techniquesmentioning

confidence: 99%

Review on Modeling and Control of Flexible Link Manipulators

Subedi¹,

Tyapin²,

Hovland³

2020

MIC

View full text Add to dashboard Cite

show abstract

“…The resulting closed-loop overall system, after substituting of (37) (38) where q ∈ R n and z ∈ R n represent the link angles and elastic torque. The first simulation presented in Fig.…”

Section: A Design Examplementioning

confidence: 99%

Reconfigurable control of flexible joint robot with actuator fault and uncertainty

Elghoul

Tellili

Abdelkrim

2019

Journal of Electrical Engineering

View full text Add to dashboard Cite

This paper presents the fault tolerant control (FTC) of a flexible joint robot using singular perturbation method in order to compensate for the lost performance due to the occurrence of actuator fault and the uncertainty. This FTC is based on Lyapunov redesign principle. The singular perturbation method is used to reduce the dynamic model of the flexible joint robot in a fast and slow subsystem. The time scale reduction of the flexible joint model is carried out when their joint stiffness is large enough and the singular perturbation parameter is set to zero. The fault-tolerant control structure in this paper is based on two parts. The first term described the composite control for the system without defect and without uncertainty which represents the sum between slow and fast controllers. While the second term of the fault tolerant command describes additive control designed to compensate for the fault effect of the actuator on the uncertain system. The additive approach is based on the Lyapunov theorem, which guarantees asymptotic stability despite the presence of actuator defects and the parametric uncertainty. The theoretical results are applied on a robot manipulator with a single flexible joint.

show abstract

“…Due to the limitations of the above methods and the amazing self-learning ability of intelligent control, researchers have shown great interest in developing learning-based control methods such as adaptive neural network control [13]- [18] and reinforcement learning (RL) control [19]- [23]. And adaptive neural network control has been successfully applied to multilink robots [15], [16], biped robots [17] and marine vessels [18].…”

Section: Introductionmentioning

confidence: 99%

Multi Pseudo Q-Learning-Based Deterministic Policy Gradient for Tracking Control of Autonomous Underwater Vehicles

Shi

Song

et al. 2019

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

This paper investigates trajectory tracking problem for a class of underactuated autonomous underwater vehicles (AUVs) with unknown dynamics and constrained inputs. Different from existing policy gradient methods which employ single actor-critic but cannot realize satisfactory tracking control accuracy and stable learning, our proposed algorithm can achieve high-level tracking control accuracy of AUVs and stable learning by applying a hybrid actors-critics architecture, where multiple actors and critics are trained to learn a deterministic policy and action-value function, respectively. Specifically, for the critics, the expected absolute Bellman error based updating rule is used to choose the worst critic to be updated in each time step. Subsequently, to calculate the loss function with more accurate target value for the chosen critic, Pseudo Q-learning, which uses sub-greedy policy to replace the greedy policy in Q-learning, is developed for continuous action spaces, and Multi Pseudo Q-learning (MPQ) is proposed to reduce the overestimation of action-value function and to stabilize the learning. As for the actors, deterministic policy gradient is applied to update the weights, and the final learned policy is defined as the average of all actors to avoid large but bad updates. Moreover, the stability analysis of the learning is given qualitatively. The effectiveness and generality of the proposed MPQ-based Deterministic Policy Gradient (MPQ-DPG) algorithm are verified by the application on AUV with two different reference trajectories. And the results demonstrate high-level tracking control accuracy and stable learning of MPQ-DPG. Besides, the results also validate that increasing the number of the actors and critics will further improve the performance.

show abstract

Reinforcement learning control of a single‐link flexible robotic manipulator

Cited by 62 publications

References 50 publications

Review on Modeling and Control of Flexible Link Manipulators

Review on Modeling and Control of Flexible Link Manipulators

Reconfigurable control of flexible joint robot with actuator fault and uncertainty

Multi Pseudo Q-Learning-Based Deterministic Policy Gradient for Tracking Control of Autonomous Underwater Vehicles

Contact Info

Product

Resources

About