Adaptive stiffness control of passivity-based biped robot on compliant ground using double deep Q network

Wu, Yao; Yao, Daojin; Guo, Zhao; Xiao, Xiaohui

doi:10.1177/0954406218781402

Cited by 7 publications

(7 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…After training process, the trained DQN networks would be used as hip impulse torque controller. Combined with the hip stiffness modulation, the biped robot was controlled under proposed controller as equation (31). To evaluate the performance of proposed chaotic controller, four experimental tests were conducted as follows.…”

Section: Controller Setup and Training Resultsmentioning

confidence: 99%

“…30 Besides, adaptive model-free controllers using RL were proposed to stabilize PDW-based robots, improving robustness to disturbance and time varying environments, while maintain PDW's merits of human-like motion and high energy efficiency. 31,32 So this paper expanded out previous work to improve the PDWbased robot's versatility by stabilizing its chaotic behaviors.…”

Section: Introductionmentioning

confidence: 93%

“…In our previous work, it had been proved that hip stiffness could be modulated to improve robustness and versatility of PDW-based robot. 31 So hip stiffness k h in compliant biped robot was set as accessible parameter. The Poincaré return map as equation (10) under variable hip stiffness could be rewritten as:…”

Section: Chaotic Gait Controllermentioning

confidence: 99%

See 2 more Smart Citations

A hybrid chaotic controller integrating hip stiffness modulation and reinforcement learning-based torque control to stabilize passive dynamic walking

Qiao

Yao

2022

Proceedings of the Institution of Mechanical Engineers, Part C:

Self Cite

View full text Add to dashboard Cite

Implementation of bipedal stable walking has attracted a lot of interest. Passive dynamic walking (PDW) is one promising manner to generate natural bipedal walking with high energy efficiency. However, how to improve stability and versatility of PDW-based robot against disturbance and time varying environments is still a big challenge in robotics. Chaos and bifurcations are intrinsic features of biped dynamic walking, which are important reasons for the failure of PDW. Thus a hybrid chaotic controller to stabilize chaos and bifurcations of PDW was proposed in this paper. At first, the dynamics model of compliant biped robot was set up, and routine to bipedal chaotic motions was found through parameter study. Then one hybrid chaotic controller integrating hip stiffness modulation and hip impulse torque control was developed, where hip impulse torque control based on reinforcement learning was trained to get state variables close to fixed point of PDW, and then hip stiffness modulation was conducted based on Ott-Grebogi-Yorke method to stabilize unstable motions of fixed point. Simulation results showed that period-1 stable walking could be gained for biped robots from chaotic motions, against original value disturbance, force disturbance and in time varying environments. The proposed hybrid chaotic controller could be used to stabilize bipedal chaotic motions and make the passivity-based robot become robust and versatile to disturbed and time changing environments.

show abstract

Section: Controller Setup and Training Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 93%

See 1 more Smart Citation

A hybrid chaotic controller integrating hip stiffness modulation and reinforcement learning-based torque control to stabilize passive dynamic walking

Qiao

Yao

2022

Proceedings of the Institution of Mechanical Engineers, Part C:

Self Cite

View full text Add to dashboard Cite

show abstract

“…Some of the recent optimization approaches for controlling robot locomotion were designed to respond adaptively to environmental changes, unknown initial conditions, and/or damage. Examples include stiffness control of a simulated model of a passive bipedal robot using a double-deep Q network presented in ( Wu et al., 2019 ) and intelligent trial and error algorithm for robots to adapt to damage presented in ( Cully et al., 2015 ). Both algorithms require offline training.…”

Section: Related Workmentioning

confidence: 99%

An online learning algorithm for adapting leg stiffness and stride angle for efficient quadruped robot trotting

et al. 2023

View full text Add to dashboard Cite

Animals adjust their leg stiffness and stride angle in response to changing ground conditions and gait parameters, resulting in improved stability and reduced energy consumption. This paper presents an online learning algorithm that attempts to mimic such animal behavior by maximizing energy efficiency on the fly or equivalently, minimizing the cost of transport of legged robots by adaptively changing the leg stiffness and stride angle while the robot is traversing on grounds with unknown characteristics. The algorithm employs an approximate stochastic gradient method to change the parameters in real-time, and has the following advantages: (1) the algorithm is computationally efficient and suitable for real-time operation; (2) it does not require training; (3) it is model-free, implying that precise modeling of the robot is not required for good performance; and (4) the algorithm is generally applicable and can be easily incorporated into a variety of legged robots with adaptable parameters and gaits beyond those implemented in this paper. Results of exhaustive performance assessment through numerical simulations and experiments on an under-actuated quadruped robot with compliant legs are included in the paper. The robot platform used a pneumatic piston in each leg as a variable, passive compliant element. Performance evaluation using simulations and experiments indicated that the algorithm was capable of converging to near-optimal values of the cost of transport for given operating conditions, terrain properties, and gait characteristics with no prior knowledge of the terrain and gait conditions. The simplicity of the algorithm and its demonstrably improved performance make the approach of this paper an excellent candidate for adaptively controlling tunable parameters of compliant, legged robots.

show abstract

“…Previous research in legged systems has indicated that leg stiffness is crucial for enhancing versatility over compliant terrains, as well as regulating external disturbances and improving energy efficiency [13][14][15][16]. Furthermore, research in biomechanics has shown that humans and birds adjust leg stiffness based on ground stiffness [17][18][19][20].…”

Section: Introductionmentioning

confidence: 99%

Robust Dynamic Walking for a 3D Dual-SLIP Model under One-Step Unilateral Stiffness Perturbations: Towards Bipedal Locomotion over Compliant Terrain

Karakasis¹,

Poulakakis²,

Artemiadis³

2022

Preprint

View full text Add to dashboard Cite

Bipedal walking is one of the most important hallmarks of human that robots have been trying to mimic for many decades. Although previous control methodologies have achieved robot walking on some terrains, there is a need for a framework allowing stable and robust locomotion over a wide range of compliant surfaces. This work proposes a novel biomechanics-inspired controller that adjusts the stiffness of the legs in support for robust and dynamic bipedal locomotion over compliant terrains. First, the 3D Dual-SLIP model is extended to support for the first time locomotion over compliant surfaces with variable stiffness and damping parameters. Then, the proposed controller is compared to a Linear-Quadratic Regulator (LQR) controller, in terms of robustness on stepping on soft terrain. The LQR controller is shown to be robust only up to a moderate ground stiffness level of 174 kN/m, while it fails in lower stiffness levels. On the contrary, the proposed controller can produce stable gait in stiffness levels as low as 30 kN/m, which results in a vertical ground penetration of the leg that is deeper than 10% of its rest length. The proposed framework could advance the field of bipedal walking, by generating stable walking trajectories for a wide range of compliant terrains useful for the control of bipeds and humanoids, as well as by improving controllers for prosthetic devices with tunable stiffness.

show abstract

Adaptive stiffness control of passivity-based biped robot on compliant ground using double deep Q network

Cited by 7 publications

References 34 publications

A hybrid chaotic controller integrating hip stiffness modulation and reinforcement learning-based torque control to stabilize passive dynamic walking

A hybrid chaotic controller integrating hip stiffness modulation and reinforcement learning-based torque control to stabilize passive dynamic walking

An online learning algorithm for adapting leg stiffness and stride angle for efficient quadruped robot trotting

Robust Dynamic Walking for a 3D Dual-SLIP Model under One-Step Unilateral Stiffness Perturbations: Towards Bipedal Locomotion over Compliant Terrain

Contact Info

Product

Resources

About