Shaping in Practice: Training Wheels to Learn Fast Hopping Directly in Hardware

Heim, Steve; Ruppert, Felix; Sarvestani, Alborz Aghamaleki; Spröwitz, Alexander

doi:10.1109/icra.2018.8460984

Cited by 8 publications

(7 citation statements)

References 18 publications

(18 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The knee joint stiffness is realized by a spring that wraps around the knee joint on a cam mechanism, inspired by a knee cap (patella) (Allen et al, 2017; Heim et al, 2018). The knee cam mechanism linearizes the knee spring deflection over knee angle.…”

Section: Methodsmentioning

confidence: 99%

Series Elastic Behavior of Biarticular Muscle-Tendon Structure in a Robotic Leg

Ruppert

Badri-Spröwitz

2019

Front. Neurorobot.

Self Cite

View full text Add to dashboard Cite

We investigate the role of lower leg muscle-tendon structures in providing serial elastic behavior to the hip actuator. We present a leg design with physical elastic elements in leg angle and virtual leg axis direction, and its impact onto energy efficient legged locomotion. By testing and comparing two robotic lower leg spring configurations, we can provide potential explanations of the functionality of similar animal leg morphologies with lower leg muscle-tendon network structures. We investigate the effects of leg angle compliance during locomotion. In a proof of concept, we show that a leg with a gastrocnemius inspired elasticity possesses elastic components that deflect in leg angle directions. The leg design with elastic elements in leg angle direction can store hip actuator energy in the series elastic element. We then show the leg's advantages in mechanical design in a vertical drop experiment. In the drop experiments the biarticular leg requires 46% less power. During drop loading, the leg adapts its posture and stores the energy in its springs. The increased energy storing capacity in leg angle direction reduces energy requirements and cost of transport by 31% during dynamic hopping to a cost of transport of 1.2 at 0.9 kg body weight. The biarticular robot leg design has major advantages, especially compared to more traditional robot designs. Despite its high degree of under-actuation, it is easy to converge into and maintain dynamic hopping locomotion. The presented control is based on a simple-to-implement, feed-forward pattern generator. The biarticular legs lightweight design can be rapidly assembled and is largely made from elements created by rapid prototyping. At the same time it is robust, and passively withstands drops from 200% body height. The biarticular leg shows, to the best of the authors' knowledge, the lowest achieved relative cost of transport documented for all dynamically hopping and running robots of 64% of a comparable natural runner's COT.

show abstract

Section: Methodsmentioning

confidence: 99%

Series Elastic Behavior of Biarticular Muscle-Tendon Structure in a Robotic Leg

Ruppert

Badri-Spröwitz

2019

Front. Neurorobot.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Randlov proved that for a finite Markovian decision process with a limited reward signal, it is guaranteed that if a series of tasks converges to the original one, then the optimal value function converges to the original one as well [25]. In [26], a temporary device to reduce gravity helped the learning of single-leg hopping, showcasing the potential of shaping in real-world applications. The same concept is also known by the term curriculum learning in RL [27].…”

Section: Introductionmentioning

confidence: 99%

Learning to stop: a unifying principle for legged locomotion in varying environments

et al. 2021

View full text Add to dashboard Cite

Evolutionary studies have unequivocally proven the transition of living organisms from water to land. Consequently, it can be deduced that locomotion strategies must have evolved from one environment to the other. However, the mechanism by which this transition happened and its implications on bio-mechanical studies and robotics research have not been explored in detail. This paper presents a unifying control strategy for locomotion in varying environments based on the principle of ‘learning to stop’. Using a common reinforcement learning framework, deep deterministic policy gradient, we show that our proposed learning strategy facilitates a fast and safe methodology for transferring learned controllers from the facile water environment to the harsh land environment. Our results not only propose a plausible mechanism for safe and quick transition of locomotion strategies from a water to land environment but also provide a novel alternative for safer and faster training of robots.

show abstract

“…Despite the good results in the simulated environments and the large application of the actor-critic approach, some authors emphasize the challenge of implementing RL algorithms directly on the real plant [24]. The full control of the vehicle commands for example can generate unsafe behaviors as accelerating the vehicle against obstacles, or driving the vehicle to its dynamic limits, causing accidents.…”

Section: Introductionmentioning

confidence: 99%

Actor-Critic Traction Control Based on Reinforcement Learning with Open-Loop Training

Drechsler

Fiorentin

Göllinger

2021

Modelling and Simulation in Engineering

View full text Add to dashboard Cite

The use of actor-critic algorithms can improve the controllers currently implemented in automotive applications. This method combines reinforcement learning (RL) and neural networks to achieve the possibility of controlling nonlinear systems with real-time capabilities. Actor-critic algorithms were already applied with success in different controllers including autonomous driving, antilock braking system (ABS), and electronic stability control (ESC). However, in the current researches, virtual environments are implemented for the training process instead of using real plants to obtain the datasets. This limitation is given by trial and error methods implemented for the training process, which generates considerable risks in case the controller directly acts on the real plant. In this way, the present research proposes and evaluates an open-loop training process, which permits the data acquisition without the control interaction and an open-loop training of the neural networks. The performance of the trained controllers is evaluated by a design of experiments (DOE) to understand how it is affected by the generated dataset. The results present a successful application of open-loop training architecture. The controller can maintain the slip ratio under adequate levels during maneuvers on different floors, including grounds that are not applied during the training process. The actor neural network is also able to identify the different floors and change the acceleration profile according to the characteristics of each ground.

show abstract

Shaping in Practice: Training Wheels to Learn Fast Hopping Directly in Hardware

Cited by 8 publications

References 18 publications

Series Elastic Behavior of Biarticular Muscle-Tendon Structure in a Robotic Leg

Series Elastic Behavior of Biarticular Muscle-Tendon Structure in a Robotic Leg

Learning to stop: a unifying principle for legged locomotion in varying environments

Actor-Critic Traction Control Based on Reinforcement Learning with Open-Loop Training

Contact Info

Product

Resources

About