DeepGait: Planning and Control of Quadrupedal Gaits using Deep Reinforcement Learning

Tsounis, Vassilios; Alge, Mitja; Farshidian, Farbod; Hutter, Marco

doi:10.48550/arxiv.1909.08399

Cited by 2 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, researchers have demonstrated agile locomotion with quadrupedal robots using a state-machine [11], impulse scaling [42], and convex model predictive control [34]. The ANYmal robot [30] plans footsteps based on the inverted pendulum model [44], which is further modulated by a vision component [51]. Similarly, bipedal robots can be controlled by the fast online trajectory optimization [7] or whole-body control [35].…”

Section: Related Workmentioning

confidence: 99%

Learning to Walk in the Real World with Minimal Human Effort

Ha,

Xu,

Tan

et al. 2020

Preprint

View full text Add to dashboard Cite

Reliable and stable locomotion has been one of the most fundamental challenges for legged robots. Deep reinforcement learning (deep RL) has emerged as a promising method for developing such control policies autonomously. In this paper, we develop a system for learning legged locomotion policies with deep RL in the real world with minimal human effort.The key difficulties for on-robot learning systems are automatic data collection and safety. We overcome these two challenges by developing a multi-task learning procedure, an automatic reset controller, and a safety-constrained RL framework. We tested our system on the task of learning to walk on three different terrains: flat ground, a soft mattress, and a doormat with crevices. Our system can automatically and efficiently learn locomotion skills on a Minitaur robot with little human intervention. The supplementary video can be found at: https://youtu.be/cwyiq6dCgOc.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning to Walk in the Real World with Minimal Human Effort

Ha,

Xu,

Tan

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Yang et al [17] learn low-dimensional pattern generator parameters and use them for high-level navigation. Tsounis et al [3] use a learned high-level controller to decide a footstep location for a learned low-level policy. Hierarchical approaches have also been demonstrated on real robots, for example Li et al [2] use a high-level controller to sequence a set of pre-learned primitives and Nachum et al [1] use the high-level policy to define sub-goals for the low-level policy.…”

Section: Background and Related Workmentioning

confidence: 99%

“…In contrast, learning-based approaches do not make strict assumptions about dynamics, but are sample-inefficient to train. As a result, several works propose to leverage hierarchy as a structure to learn locomotion skills scalable to real robots [1,2,3]. However, typically in hierarchical control literature, the action space used by the high-level controller to interact with the low-level controller is user-defined [2,3,4].…”

Section: Introductionmentioning

confidence: 99%

“…As a result, several works propose to leverage hierarchy as a structure to learn locomotion skills scalable to real robots [1,2,3]. However, typically in hierarchical control literature, the action space used by the high-level controller to interact with the low-level controller is user-defined [2,3,4]. Such a user-defined action space can potentially be too restrictive for some tasks.…”

Section: Introductionmentioning

confidence: 99%

“…Such a user-defined action space can potentially be too restrictive for some tasks. For example, Tsounis et al [3] constrain the space to be a particular footstep pattern, while a different gait might move faster; Frans et al [4], Li et al [2] constrain the low-level to be a pre-defined set of primitives, constraining performance to the quality and number of primitives.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Planning in Learned Latent Action Spaces for Generalizable Legged Locomotion

Calandra

Pathak

et al. 2020

Preprint

View full text Add to dashboard Cite

Hierarchical learning has been successful at learning generalizable locomotion skills on walking robots in a sample-efficient manner. However, the lowdimensional "latent" action used to communicate between different layers of the hierarchy is typically user-designed. In this work, we present a fully-learned hierarchical framework, that is capable of jointly learning the low-level controller and the high-level action space. Next, we plan over latent actions in a model-predictive control fashion, using a learned high-level dynamics model. This framework is generalizable to multiple robots, and we present results on a Daisy hexapod simulation, A1 quadruped simulation, and Daisy robot hardware. We compare a range of learned hierarchical approaches, and show that our framework is more reliable, versatile and sample-efficient. In addition to learning approaches, we also compare to an inverse-kinematics (IK) based footstep planner, and show that our fully-learned framework is competitive in performance with IK under normal conditions, and outperforms it in adverse settings. Our hardware experiments show the Daisy hexapod achieving multiple locomotion tasks, such as goal reaching, trajectory and velocity tracking in an unstructured outdoor setting, with only 2000 hardware samples.

show abstract

DeepGait: Planning and Control of Quadrupedal Gaits using Deep Reinforcement Learning

Cited by 2 publications

References 0 publications

Learning to Walk in the Real World with Minimal Human Effort

Learning to Walk in the Real World with Minimal Human Effort

Planning in Learned Latent Action Spaces for Generalizable Legged Locomotion

Contact Info

Product

Resources

About