Learning Predictive Representations for Deformable Objects Using Contrastive Estimation

Abstract

Fig. 1: Example trajectories using our contrastive forward model for rope and cloth manipulation. The top two rows show rope manipulation from different start states to different goal states, while the bottom two rows show cloth manipulation using different colored cloths. Note that in the last row, the robot is manipulating a white cloth, but are method is able to still use a blue cloth as the goal image.

Unsupervised Feature Learning for Manipulation with Contrastive Domain Randomization

“…Contrastive learning methods, which represent spatial/temporal patterns in the data as latent vectors, are particularly useful for model-based planning [37], [38]. For example, recent methods applied MPC for cloth and rope manipulation, using a forward model learned using contrastive learning [11], [39]. Our work extends these approaches to planning that is invariant to irrelevant visual properties.…”

“…A common choice is the weighted dot-product exp(z T i W z j ) or σ(z T i W z j ) [7], [25]. The cosine distance and L2 distance have also proven effective [5], [11], [40].…”

“…However, collecting a diverse set of self-supervised robot interactions requires significant resources [8]- [10]. Exploiting the recent advances in physical simulators, Yan et al proposed to learn physical features using contrastive learning from simulated interaction data [11]. Since simulated data is easy to generate, and no reward function needs to be manually defined for unsupervised learning, the approach in [11] offers a principled and scalable method for learning general physical features, such as the configuration of nonrigid objects.…”

“…To robustify learning, a popular trick is domain randomization (DR) -randomly modify irrelevant visual properties such as lighting and object textures during training, with the hope of learning features that are invariant to these perturbations. Indeed, [11] demonstrated successful sim-to-real transfer using DR. Conversely, however, in this work we claim that a straightforward application of DR to contrastive learning is fundamentally flawed: by carefully formulating the problem, we show that the standard CPC loss function aims to maximize information between the features and both relevant and irrelevant visual properties, and therefore does not learn features that are robustly invariant to the irrelevant randomization. Indeed, the experiments in [11] were designed with a very small sim-to-real gap, and we demonstrate that when this gap is increased, the quality of the features degrade significantly.…”

“…Indeed, [11] demonstrated successful sim-to-real transfer using DR. Conversely, however, in this work we claim that a straightforward application of DR to contrastive learning is fundamentally flawed: by carefully formulating the problem, we show that the standard CPC loss function aims to maximize information between the features and both relevant and irrelevant visual properties, and therefore does not learn features that are robustly invariant to the irrelevant randomization. Indeed, the experiments in [11] were designed with a very small sim-to-real gap, and we demonstrate that when this gap is increased, the quality of the features degrade significantly. Our derivation also prescribes a simple fix -by applying a different randomization to past and future observations, which can easily be done in simulation, we guarantee that invariant representations are learned.…”