Meta-Learning Deep Visual Words for Fast Video Object Segmentation

Abstract

Human Horse Figure 1: Video object segmentation using a dictionary of deep visual words. Our proposed method represents an object as a set of cluster centroids in a learned embedding space, or "visual words", which correspond to object parts in image space (bottom row). This representation allows more robust and efficient matching as shown by our results (top row). The visual words are learned in an unsupervised manner, using meta-learning to ensure the training and inference procedures are identical. The t-SNE plot [29] on the right shows how different object parts cluster in different regions of the embedding space, and thus how our representation captures the multi-modal distribution of pixels constituting an object.

Learning to Learn Morphological Inflection for Resource-Poor Languages

“…Meta-learning techniques have been applied to a variety of machine learning problems. Outside of NLP, successful applications include image recognition (Ravi and Larochelle 2016;Vinyals et al 2016) and video object segmentation (Yang et al 2018;Behl, Najafi, and Torr 2018). Within NLP, meta-learning approaches are more scarce.…”

Learning What to Learn for Video Object Segmentation

“…Meta-learning for VOS: Since the VOS task itself includes a few-shot learning problem, it can be addressed with techniques developed for meta-learning [10,3,16]. A few recent attempts follow this direction [19,1]. The method [1] learns a classifier using k-means clustering of segmentation features in the train frame.…”

“…A few recent attempts follow this direction [19,1]. The method [1] learns a classifier using k-means clustering of segmentation features in the train frame. In [19], the final layer of a segmentation network is predicted by closed-form ridge regression [3], using the reference example pair.…”

“…The main advantage of the optimization-based formulation (1) is that the target model is predicted by directly minimizing the segmentation error in the first frame. It is thus capable of learning a powerful model, which can generate a robust prediction of the target in the coming frames.…”

“…Unlike the formulation in (1), the encoded ground-truth mask E θ (y 0 ) can be multi-dimensional. This allows the target module T τ to predict a richer representation of the object, providing powerful cues to the segmentation network.…”