Position/force control of robot manipulators using reinforcement learning

Perrusquía, Adolfo; Yu, Wen; Soria, Alberto

doi:10.1108/ir-10-2018-0209

Cited by 54 publications

(25 citation statements)

References 24 publications

(28 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To improve the controller, prior data information can be used as the initial condition for the value function to avoid scratch learning. The use of a long short‐ term memory like eligibility traces can accelerate the convergence. Optimization procedures can be used to find the optimal number of k ‐nearest neighbors.…”

Section: Simulation and Experimentsmentioning

confidence: 99%

Robust control under worst‐case uncertainty for unknown nonlinear systems using modified reinforcement learning

Perrusquía

2020

Intl J Robust & Nonlinear

Self Cite

View full text Add to dashboard Cite

Reinforcement learning (RL) is an effective method for the design of robust controllers of unknown nonlinear systems. Normal RLs for robust control, such as actor-critic (AC) algorithms, depend on the estimation accuracy. Uncertainty in the worst case requires a large state-action space, this causes overestimation and computational problems. In this article, the RL method is modified with the k-nearest neighbor and the double Q-learning algorithm. The modified RL does not need the neural estimator as AC and can stabilize the unknown nonlinear system under the worst-case uncertainty. The convergence property of the proposed RL method is analyzed. The simulations and the experimental results show that our modified RLs are much more robust compared with the classic controllers, such as the proportional-integral-derivative, the sliding mode, and the optimal linear quadratic regulator controllers. K E Y W O R D Sk-nearest neighbors, double estimator, overestimation, robust reward, state-action space, worst-case uncertainty INTRODUCTIONThe objective of robust control is to achieve robust performance in presence of disturbances. Most robust controllers are inspired by the optimal control theory, such as  2 control, 1,2 which minimizes a certain cost function to find an optimal controller. The most popular  2 controller is the linear quadratic regulator (LQR). 3 It does not work well in presence of disturbances. The  ∞ control can find a robust controller when the system has disturbances. Its performance is poor compared with the  2 control. 4 The combination of  2 and  ∞ , called  2 ∕ ∞ control, has both advantages, that is., it has optimal performance with bounded disturbances. 5 The  2 ∕ ∞ controller design needs a complete knowledge of the system dynamics. 6 These controllers are model-based. The time-varying quadratic optimization can be calculated by the zeroing neural network. It can simultaneously achieve the finite-time convergence and inherent noise tolerance. 7 However, it requires prefect activation functions. Model-free controllers, like the proportional-integral-derivative (PID) control, 8,9 the sliding mode control (SMC), 10 neural control, 11-16 among others, do not require dynamic knowledge of the system. However, parameter tuning and some prior knowledge of the disturbances prevent these model-free controllers to perform optimally.Reinforcement learning (RL) 17,18 is another effective method without models. It is designed in the sense of  2 control. 19 The temporal difference (TD) rule, such as Q-learning, is applied to find an optimal solution for Markov decision processes. 20 The advantage of RL over the other model-free methods is that it can reach optimal performances. Recent results show that RL methods can learn  2 and  ∞ controllers without system dynamics. 21 2920The main objective of RL for  2 and  ∞ is to minimize the total accumulative reward. For a robust controller, the reward is designed in the sense of control problems  2 and  ∞ . This robust reward can be static or dynam...

show abstract

Section: Simulation and Experimentsmentioning

confidence: 99%

Robust control under worst‐case uncertainty for unknown nonlinear systems using modified reinforcement learning

Perrusquía

2020

Intl J Robust & Nonlinear

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, these controllers are avoided because they lose controllability at singularity points [20,21], that is, the Jacobian loses rank. Furthermore, the contact between the slider and the workpiece generates an external force [22] which must be taken into account in the controller design [23,24] by means of the virtual work principle.…”

Section: Introductionmentioning

confidence: 99%

Optimal sliding mode control for cutting tasks of quick-return mechanisms

Perrusquía

Flores-Campos

2022

ISA Transactions

Self Cite

View full text Add to dashboard Cite

A solution of the constant cutting velocity problem of quick-return mechanisms is the main concern of this paper. An optimal sliding mode control in the task space is used to achieve uniform and accurate cuts throughout the workpiece. The switching hyperplane is designed to minimize the position error of the slider-dynamics in an infinite horizon.A Jacobian compensator is used to exploit the mechanical advantage and ensure controllability. The velocity profile is constructed in terms of the mechanism and workpiece geometric properties. Stability of the closed-loop dynamics is verified with the Lyapunov stability theory. Experiments are carried out in a quick-return mechanism prototype to validate the proposal.

show abstract

“…Traditional manipulators are mostly used in industrial production, working in a structured environment on a given trajectory to complete specific tasks. To ensure excellent tracking performance and safety, manipulators generally adopt the design scheme of rigid mechanical structure and active compliance control such as position/force [1,2] or impedance/admittance control [3,4]. However, rigid manipulators based on active compliance control still present stiffness [5] when suffering sudden impacts or handling contact transients due to the control delay, which can cause damage to itself or surrounding environments.…”

Section: Introductionmentioning

confidence: 99%

Mechanical design and analysis of a novel variable stiffness actuator with symmetrical pivot adjustment

2021

View full text Add to dashboard Cite

The safety of human-robot interaction is an essential requirement for designing collaborative robotics. Thus, this paper aims to design a novel variable stiffness actuator (VSA) that can provide safer physical human-robot interaction for collaborative robotics. VSA follows the idea of modular design, mainly including a variable stiffness module and a drive module. The variable stiffness module transmits the motion from the drive module in a roundabout manner, making the modularization of VSA possible. As the key component of the variable stiffness module, a stiffness adjustment mechanism with a symmetrical structure is applied to change the positions of a pair of pivots in two levers linearly and simultaneously, which can eliminate the additional bending moment caused by the asymmetric structure. The design of the double-deck grooves in the lever allows the pivot to move freely in the groove, avoiding the geometric constraint between the parts. Consequently, the VSA stiffness can change from zero to infinity as the pivot moves from one end of the groove to the other. To facilitate building a manipulator in the future, an expandable electrical system with a distributed structure is also proposed. Stiffness calibration and control experiments are performed to evaluate the physical performance of the designed VSA. Experiment results show that the VSA stiffness is close to the theoretical design stiffness. Furthermore, the VSA with a proportional-derivative feedback plus feedforward controller exhibits a fast response for stiffness regulation and a good performance for position tracking.

show abstract

Position/force control of robot manipulators using reinforcement learning

Cited by 54 publications

References 24 publications

Robust control under worst‐case uncertainty for unknown nonlinear systems using modified reinforcement learning

Robust control under worst‐case uncertainty for unknown nonlinear systems using modified reinforcement learning

Optimal sliding mode control for cutting tasks of quick-return mechanisms

Mechanical design and analysis of a novel variable stiffness actuator with symmetrical pivot adjustment

Contact Info

Product

Resources

About