CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning

Ahmed, Ossama Ashraf; Träuble, Frederik; Goyal, Anirudh; Neitz, Alexander; Bengio, Yoshua; Schölkopf, Bernhard; Wüthrich, Manuel; Bauer, Sebastian

doi:10.48550/arxiv.2010.04296

Cited by 22 publications

(34 citation statements)

References 31 publications

(39 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Benchmark datasets play an important role in developing machine learning methodologies. Examples include ImageNet (Deng et al, 2009) or MSCOCO (Lin et al, 2014) for computer vision, as well as cart-pole (Barto et al, 1983) or reinforcement learning (Ahmed et al, 2020).…”

Section: Related Workmentioning

confidence: 99%

GeneDisco: A Benchmark for Experimental Design in Drug Discovery

Mehrjou¹,

Soleymani²,

Jesson³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

In vitro cellular experimentation with genetic interventions, using for example CRISPR technologies, is an essential step in early-stage drug discovery and target validation that serves to assess initial hypotheses about causal associations between biological mechanisms and disease pathologies. With billions of potential hypotheses to test, the experimental design space for in vitro genetic experiments is extremely vast, and the available experimental capacity -even at the largest research institutions in the world -pales in relation to the size of this biological hypothesis space. Machine learning methods, such as active and reinforcement learning, could aid in optimally exploring the vast biological space by integrating prior knowledge from various information sources as well as extrapolating to yet unexplored areas of the experimental design space based on available data. However, there exist no standardised benchmarks and data sets for this challenging task and little research has been conducted in this area to date. Here, we introduce GeneDisco, a benchmark suite for evaluating active learning algorithms for experimental design in drug discovery. GeneDisco contains a curated set of multiple publicly available experimental data sets as well as open-source implementations of state-of-the-art active learning policies for experimental design and exploration.

show abstract

Section: Related Workmentioning

confidence: 99%

GeneDisco: A Benchmark for Experimental Design in Drug Discovery

Mehrjou¹,

Soleymani²,

Jesson³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…We evaluate and analyze our proposed MOC DRL on the CausalWorld [Ahmed et al, 2020], as this environment enables us to easily design and test different types of curricula in a fine-grained manner. It should be noted that we do not utilize any causal elements of the environment.…”

Section: Methodsmentioning

confidence: 99%

Learning Multi-Objective Curricula for Deep Reinforcement Learning

Kang¹,

Liu²,

Gupta³

et al. 2021

Preprint

View full text Add to dashboard Cite

Various automatic curriculum learning (ACL) methods have been proposed to improve the sample efficiency and final performance of deep reinforcement learning (DRL). They are designed to control how a DRL agent collects data, which is inspired by how humans gradually adapt their learning processes to their capabilities. For example, ACL can be used for subgoal generation, reward shaping, environment generation, or initial state generation. However, prior work only considers curriculum learning following one of the aforementioned predefined paradigms. It is unclear which of these paradigms are complementary, and how the combination of them can be learned from interactions with the environment. Therefore, in this paper, we propose a unified automatic curriculum learning framework to create multi-objective but coherent curricula that are generated by a set of parametric curriculum modules. Each curriculum module is instantiated as a neural network and is responsible for generating a particular curriculum. In order to coordinate those potentially conflicting modules in unified parameter space, we propose a multi-task hyper-net learning framework that uses a single hyper-net to parameterize all those curriculum modules. In addition to existing hand-designed curricula paradigms, we further design a flexible memory mechanism to learn an abstract curriculum, which may otherwise be difficult to design manually. We evaluate our method on a series of robotic manipulation tasks and demonstrate its superiority over other state-of-the-art ACL methods in terms of sample efficiency and final performance.

show abstract

“…One of the biggest challenges in reinforcement learning (RL) is the brittleness of trained agents to distribution shifts in the environment. Recent studies have developed benchmarks to quantify the generalization performance of RL agents in out-of-distribution environments [10,11,12]. Indeed, this problem is particularly relevant to the field of robot learning where policies are often trained in simulation and directly transferred to hardware, resulting in an OOD deployment of the policy due to the mismatch between the simulator and the real world.…”

Section: Related Workmentioning

confidence: 99%

Learning Provably Robust Motion Planners Using Funnel Libraries

Gurgen¹,

Majumdar²,

Veer³

2021

Preprint

View full text Add to dashboard Cite

This paper presents an approach for learning motion planners that are accompanied with probabilistic guarantees of success on new environments that hold uniformly for any disturbance to the robot's dynamics within an admissible set. We achieve this by bringing together tools from generalization theory and robust control. First, we curate a library of motion primitives where the robustness of each primitive is characterized by an over-approximation of the forward reachable set, i.e., a "funnel". Then, we optimize probably approximately correct (PAC)-Bayes generalization bounds for training our planner to compose these primitives such that the entire funnels respect the problem specification. We demonstrate the ability of our approach to provide strong guarantees on two simulated examples: (i) navigation of an autonomous vehicle under external disturbances on a five-lane highway with multiple vehicles, and (ii) navigation of a drone across an obstacle field in the presence of wind disturbances.

show abstract

CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning

Cited by 22 publications

References 31 publications

GeneDisco: A Benchmark for Experimental Design in Drug Discovery

GeneDisco: A Benchmark for Experimental Design in Drug Discovery

Learning Multi-Objective Curricula for Deep Reinforcement Learning

Learning Provably Robust Motion Planners Using Funnel Libraries

Contact Info

Product

Resources

About