A Comparison of Action Spaces for Learning Manipulation Tasks

Varin, Patrick; Grossman, Lev; Kuindersma, Scott

doi:10.48550/arxiv.1908.08659

Cited by 1 publication

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Robot learning research has a rich history in combining well-established control methods -often in the form of an action space -with machine learning [32]. This hybrid approach can result in an improvement in sample efficiency, robustness, and quality of the learned policies [39], [45]. In [42], a dualarm system learns to juggle by learning a world model and an optimal control policy.…”

Section: Related Workmentioning

confidence: 99%

Learning Robotic Manipulation Skills Using an Adaptive Force-Impedance Action Space

Ulmer¹,

Aljalbout²,

Schwarz³

et al. 2021

Preprint

View full text Add to dashboard Cite

Intelligent agents must be able to think fast and slow to perform elaborate manipulation tasks. Reinforcement Learning (RL) has led to many promising results on a range of challenging decision-making tasks. However, in real-world robotics, these methods still struggle, as they require large amounts of expensive interactions and have slow feedback loops. On the other hand, fast human-like adaptive control methods can optimize complex robotic interactions, yet fail to integrate multimodal feedback needed for unstructured tasks. In this work, we propose to factor the learning problem in a hierarchical learning and adaption architecture to get the best of both worlds. The framework consists of two components, a slow reinforcement learning policy optimizing the task strategy given multimodal observations, and a fast, real-time adaptive control policy continuously optimizing the motion, stability, and effort of the manipulator. We combine these components through a bio-inspired action space that we call AFORCE. We demonstrate the new action space on a contact-rich manipulation task on real hardware and evaluate its performance on three simulated manipulation tasks. Our experiments show that AFORCE drastically improves sample efficiency while reducing energy consumption and improving safety.

show abstract

Section: Related Workmentioning

confidence: 99%