On Compositions of Transformations in Contrastive Self-Supervised Learning

Patrick, Mandela; Asano, Yuki; Кузнецова, П.И.; Fong, Ruth; Henriques, João F.; Zweig, Geoffrey; Vedaldi, Andrea

doi:10.48550/arxiv.2003.04298

Cited by 1 publication

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent works (Li et al, 2021b;a) apply contrastive learning over trajectories for compact representations, but ignore the influence of behavior policy mismatch. (Chen et al, 2020;He et al, 2020;Patrick et al, 2020), multimodal learning (Sermanet et al, 2018;Tian et al, 2020;Liu et al, 2020b) and image-based reinforcement learning (Anand et al, 2019;Stooke et al, 2021;Laskin et al, 2020).…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Robust Task Representations for Offline Meta-Reinforcement Learning via Contrastive Learning

Yuan¹,

Lu²

2022

Preprint

View full text Add to dashboard Cite

We study offline meta-reinforcement learning, a practical reinforcement learning paradigm that learns from offline data to adapt to new tasks. The distribution of offline data is determined jointly by the behavior policy and the task. Existing offline meta-reinforcement learning algorithms cannot distinguish these factors, making task representations unstable to the change of behavior policies. To address this problem, we propose a contrastive learning framework for task representations that are robust to the distribution mismatch of behavior policies in training and test. We design a bi-level encoder structure, use mutual information maximization to formalize task representation learning, derive a contrastive learning objective, and introduce several approaches to approximate the true distribution of negative pairs. Experiments on a variety of offline meta-reinforcement learning benchmarks demonstrate the advantages of our method over prior methods, especially on the generalization to out-of-distribution behavior policies.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Diversity: According to prior works (Patrick et al, 2020;Chen et al, 2020), the performance of contrastive learning depends on the diversity of negative pairs. More diversity increases the difficulty in optimizing InfoNCE and helps the encoder to learn meaningful representations.…”

Section: Negative Pairs Generationmentioning

confidence: 99%

Robust Task Representations for Offline Meta-Reinforcement Learning via Contrastive Learning

Yuan¹,

Lu²

2022

Preprint

View full text Add to dashboard Cite

show abstract

On Compositions of Transformations in Contrastive Self-Supervised Learning

Cited by 1 publication

References 0 publications

Robust Task Representations for Offline Meta-Reinforcement Learning via Contrastive Learning

Robust Task Representations for Offline Meta-Reinforcement Learning via Contrastive Learning

Contact Info

Product

Resources

About