The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixels

Stone, Austin V.; Ramírez, Oscar M.; Konolige, Kurt; Jonschkowski, Rico

doi:10.48550/arxiv.2101.02722

Cited by 13 publications

(42 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Setup We conduct experiments using nine domains from the DeepMind Control Suite (DMC) (Tassa et al, 2018). We use DMC as the training (source) environment and the Distracting Control Suite (DistractingCS) (Stone et al, 2021) as the distracted test (target) environment. DistractingCS adds three types of distractions to the DeepMind Control Suite in the form of changes to the background image, deviations in color, and changes to the camera pose relative to training.…”

Section: Methodsmentioning

confidence: 99%

Invariance Through Latent Alignment

Yoneda¹,

Yang²,

Walter³

et al. 2021

Preprint

View full text Add to dashboard Cite

We introduce a general approach, called Invariance through Inference, for improving the test-time performance of an agent in deployment environments with unknown perceptual variations. Instead of producing invariant visual features through interpolation, invariance through inference turns adaptation at deployment-time into an unsupervised learning problem. This is achieved in practice by deploying a straightforward algorithm that tries to match the distribution of latent features to the agent's prior experience, without relying on paired data. Although simple, we show that this idea leads to surprising improvements on a variety of adaptation scenarios without access to deploymenttime rewards, including changes in camera poses and lighting conditions. Results are presented on challenging distractor control suite, a robotics environment with image-based observations. Here is our project page: https:// invariance-through-inference.github.io/.

show abstract

Section: Methodsmentioning

confidence: 99%

Invariance Through Latent Alignment

Yoneda¹,

Yang²,

Walter³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Using CoRe, we get state-of-the-art results on the challenging Distracting Control Suite benchmark (Stone et al, 2021) which includes background, camera, and color distractions applied simultaneously.…”

Section: Core | Visualizationmentioning

confidence: 99%

“…3 EXPERIMENTS We use the Distracting Control Suite (DCS) (Stone et al, 2021) and Robosuite (Zhu et al, 2020) to benchmark our model's ability to withstand visual distractions. DCS consists of six simulated robotic control tasks derived from the DeepMind Control Suite (Tassa et al, 2018).…”

Section: Behavior Learningmentioning

confidence: 99%

Robust Robotic Control from Pixels using Contrastive Recurrent State-Space Models

Srivastava¹,

Talbott²,

Bertran³

et al. 2021

Preprint

View full text Add to dashboard Cite

Modeling the world can benefit robot learning by providing a rich training signal for shaping an agent's latent state space. However, learning world models in unconstrained environments over high-dimensional observation spaces such as images is challenging. One source of difficulty is the presence of irrelevant but hard-to-model background distractions, and unimportant visual details of taskrelevant entities. We address this issue by learning a recurrent latent dynamics model which contrastively predicts the next observation. This simple model leads to surprisingly robust robotic control even with simultaneous camera, background, and color distractions. We outperform alternatives such as bisimulation methods which impose state-similarity measures derived from divergence in future reward or future optimal actions. We obtain state-of-the-art results on the Distracting Control Suite, a challenging benchmark for pixel-based robotic control. Code for our model can be found at https://github.com/apple/ml-core.

show abstract

“…In this work, we provide a unifying view of currently used bisimulation metrics, their empirical approximations, and provide a mathematically formal way of producing unbiased sample estimates of bisimulation metrics in continuous, stochastic environments for a wide variety of desired invariance specifications. We show how this contribution can meaningfully improve results on current approximations on the Distracting Control Suite benchmark (Stone et al, 2021); as well as an experimental analysis on how bisimulation interacts with existing augmentation techniques and agent capacity.…”

Section: Introductionmentioning

confidence: 96%

Efficient Embedding of Semantic Similarity in Control Policies via Entangled Bisimulation

Bertran¹,

Talbott²,

Srivastava³

et al. 2022

Preprint

View full text Add to dashboard Cite

Learning generalizeable policies from visual input in the presence of visual distractions is a challenging problem in reinforcement learning. Recently, there has been renewed interest in bisimulation metrics as a tool to address this issue; these metrics can be used to learn representations that are, in principle, invariant to irrelevant distractions by measuring behavioural similarity between states. An accurate, unbiased, and scalable estimation of these metrics has proved elusive in continuous state and action scenarios. We propose entangled bisimulation, a bisimulation metric that allows the specification of the distance function between states, and can be estimated without bias in continuous state and action spaces. We show how entangled bisimulation can meaningfully improve over previous methods on the Distracting Control Suite (DCS), even when added on top of data augmentation techniques.

show abstract

The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixels

Cited by 13 publications

References 14 publications

Invariance Through Latent Alignment

Invariance Through Latent Alignment

Robust Robotic Control from Pixels using Contrastive Recurrent State-Space Models

Efficient Embedding of Semantic Similarity in Control Policies via Entangled Bisimulation

Contact Info

Product

Resources

About