Unsupervised Learning from Videos for Object Discovery in Single Images

Abstract

This paper proposes a method for discovering the primary objects in single images by learning from videos in a purely unsupervised manner—the learning process is based on videos, but the generated network is able to discover objects from a single input image. The rough idea is that an image typically consists of multiple object instances (like the foreground and background) that have spatial transformations across video frames and they can be sparsely represented. By exploring the sparsity representation of a video with a neural network, one may learn the features of each object instance without any labels, which can be used to discover, recognize, or distinguish object instances from a single image. In this paper, we consider a relatively simple scenario, where each image roughly consists of a foreground and a background. Our proposed method is based on encoder-decoder structures to sparsely represent the foreground, background, and segmentation mask, which further reconstruct the original images. We apply the feed-forward network trained from videos for object discovery in single images, which is different from the previous co-segmentation methods that require videos or collections of images as the input for inference. The experimental results on various object segmentation benchmarks demonstrate that the proposed method extracts primary objects accurately and robustly, which suggests that unsupervised image learning tasks can benefit from the sparsity of images and the inter-frame structure of videos.

Image Segmentation from Sparse Decomposition with a Pretrained Object-Detection Network

“…Quan et al [16] used a pretrained network to obtain the semantic features, and proposed a manifold ranking method to discover the common objects. Zhao et al [17] constrained the proportion of foreground object in the image. These methods segmented images based on hand-crafted features and unscalable prior.…”

“…More recent cosegmentation methods are formulated as modeling optimization [12], object saliency [13], or data clustering [14][15][16] problems in more than two images. However, the existing methods still suffer from certain limitations including hand-crafted features and unscalable prior [17]. Moreover, these methods rely on the collections of images to segment the objects and fail to segment in a single image.…”