Pretrained equivariant features improve unsupervised landmark discovery

Abstract

Locating semantically meaningful landmark points is a crucial component of a large number of computer vision pipelines. Because of the small number of available datasets with ground truth landmark annotations, it is important to design robust unsupervised and semisupervised methods for landmark detection. Many of the recent unsupervised learning methods rely on the equivariance properties of landmarks to synthetic image deformations. Our work focuses on such widely used methods and sheds light on its core problem, its inability to produce equivariant intermediate convolutional features. This nding leads us to formulate a two-step unsupervised approach that overcomes this challenge by rst learning powerful pixel-based features and then use the pre-trained features to learn a landmark detector by the traditional equivariance method. Our method produces state-of-the-art results in several challenging landmark detection datasets such as the BBC Pose dataset and the Cat-Head dataset. It performs comparably on a range of other benchmarks.

C2F-TCN: A Framework for Semi- and Fully-Supervised Temporal Action Segmentation

“…Contrastive learning dates back to [52] but was more recently formalized in SimCLR [18]. Most works [21], [22], [53], [54] hinge on well-defined data augmentations, with the goal of bringing together the original and augmented samples in the feature space.…”

“…Equipped with the C2F-TCN architecture and a FA strategy for learning, we formulate an unsupervised representation learning algorithm suitable for temporal action segmentation. We are hereby inspired by the success of the contrastive SimCLR framework for images [18], videos [19], [20], and other areas of machine learning [21], [22]. The standard SimCLR technique brings representations of images [18] or videos [19] close to their augmented counterparts during training.…”