Exploiting Symmetries in Reinforcement Learning of Bimanual Robotic Tasks

Amadio, Fabio; Colomé, Adrià; Torras, Carme

doi:10.1109/lra.2019.2898330

Cited by 18 publications

(6 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Introducing data augmentation into RL has primarily aimed at enhancing data efficiency [12]. A natural approach to exploit data augmentation in single-agent RL is to obtain more data via image transformation during model training [17,37,18,1]. Another type of approach introduces data augmentation through an innovative contrastive learning framework called the CURL.…”

Section: Data Augmentation In Rlmentioning

confidence: 99%

ESP: Exploiting Symmetry Prior for Multi-Agent Reinforcement Learning

Yu,

Shi,

Feng

et al. 2023

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

Multi-agent reinforcement learning (MARL) has achieved promising results in recent years. However, most existing reinforcement learning methods require a large amount of data for model training. In addition, data-efficient reinforcement learning requires the construction of strong inductive biases, which are ignored in the current MARL approaches. Inspired by the symmetry phenomenon in multi-agent systems, this paper proposes a framework for exploiting prior knowledge by integrating data augmentation and a well-designed consistency loss into the existing MARL methods. In addition, the proposed framework is model-agnostic and can be applied to most of the current MARL algorithms. Experimental tests on multiple challenging tasks demonstrate the effectiveness of the proposed framework. Moreover, the proposed framework is applied to a physical multi-robot testbed to show its superiority.

show abstract

Section: Data Augmentation In Rlmentioning

confidence: 99%

ESP: Exploiting Symmetry Prior for Multi-Agent Reinforcement Learning

Yu,

Shi,

Feng

et al. 2023

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

show abstract

“…The field of bimanual manipulation has been long studied as a problem involving both hardware design and control [45,21,59,49]. In recent years, researchers applied learning based approach to bimanual manipulation using imitation learning from demonstrations [62,17,54,60] and reinforcement learning [30,1,8,10,18]. For example, Amadio et al [1] proposed to leverage probabilistic movement primitives from human demonstrations.…”

Section: Related Workmentioning

confidence: 99%

“…In recent years, researchers applied learning based approach to bimanual manipulation using imitation learning from demonstrations [62,17,54,60] and reinforcement learning [30,1,8,10,18]. For example, Amadio et al [1] proposed to leverage probabilistic movement primitives from human demonstrations. Chitnis et al [8] further introduced a high-level planning policy to combine a set of parameterized primitives to solve complex manipulation tasks.…”

Section: Related Workmentioning

confidence: 99%

DAIR: Disentangled Attention Intrinsic Regularization for Safe and Efficient Bimanual Manipulation

Zhang,

Jian,

et al. 2021

Preprint

View full text Add to dashboard Cite

We address the problem of solving complex bimanual robot manipulation tasks on multiple objects with sparse rewards. Such complex tasks can be decomposed into sub-tasks that are accomplishable by different robots concurrently or sequentially for better efficiency. While previous reinforcement learning approaches primarily focus on modeling the compositionality of sub-tasks, two fundamental issues are largely ignored particularly when learning cooperative strategies for two robots: (i) domination, i.e., one robot may try to solve a task by itself and leaves the other idle; (ii) conflict, i.e., one robot can easily interrupt another's workspace when executing different sub-tasks simultaneously. To tackle these two issues, we propose a novel technique called disentangled attention, which provides an intrinsic regularization for two robots to focus on separate sub-tasks and objects. We evaluate our method on four bimanual manipulation tasks. Experimental results show that our proposed intrinsic regularization successfully avoids domination and reduces conflicts for the policies, which leads to significantly more effective cooperative strategies than all the baselines. Our project page with videos is at https://mehooz.github.io/bimanual-attention/. * Equal contribution.Preprint. Under review.

show abstract

“…Bimanual Robot Manipulation: Bimanual manipulation is a practical problem of great interest [31]. Reinforcement Learning (RL) has been applied to bimanual manipulation tasks [3,7,8,17], but RL methods must deal with the increased burden of exploration due to the presence of two arms. Prior work has tried to address the exploration burden by assuming access to parametrized skills such as reaching and twisting [7], by encouraging efficient exploration via intrinsic motivation [8], and leveraging movement primitives from human demonstrations [3].…”

Section: Related Workmentioning

confidence: 99%

Learning Multi-Arm Manipulation Through Collaborative Teleoperation

Tung¹,

Wong²,

Mandlekar³

et al. 2020

Preprint

View full text Add to dashboard Cite

Imitation Learning (IL) is a powerful paradigm to teach robots to perform manipulation tasks by allowing them to learn from human demonstrations collected via teleoperation, but has mostly been limited to single-arm manipulation. However, many real-world tasks require multiple arms, such as lifting a heavy object or assembling a desk. Unfortunately, applying IL to multi-arm manipulation tasks has been challenging -asking a human to control more than one robotic arm can impose significant cognitive burden and is often only possible for a maximum of two robot arms. To address these challenges, we present MULTI-ARM ROBOTURK (MART), a multi-user data collection platform that allows multiple remote users to simultaneously teleoperate a set of robotic arms and collect demonstrations for multi-arm tasks. Using MART, we collected demonstrations for five novel two and three-arm tasks from several geographically separated users. From our data we arrived at a critical insight: most multi-arm tasks do not require global coordination throughout its full duration, but only during specific moments. We show that learning from such data consequently presents challenges for centralized agents that directly attempt to model all robot actions simultaneously, and perform a comprehensive study of different policy architectures with varying levels of centralization on our tasks. Finally, we propose and evaluate a base-residual policy framework that allows trained policies to better adapt to the mixed coordination setting common in multi-arm manipulation, and show that a centralized policy augmented with a decentralized residual model outperforms all other models on our set of benchmark tasks. Additional results and videos at https://roboturk. stanford.edu/multiarm

show abstract

Exploiting Symmetries in Reinforcement Learning of Bimanual Robotic Tasks

Cited by 18 publications

References 16 publications

ESP: Exploiting Symmetry Prior for Multi-Agent Reinforcement Learning

ESP: Exploiting Symmetry Prior for Multi-Agent Reinforcement Learning

DAIR: Disentangled Attention Intrinsic Regularization for Safe and Efficient Bimanual Manipulation

Learning Multi-Arm Manipulation Through Collaborative Teleoperation

Contact Info

Product

Resources

About