Unsupervised Multi-class Joint Image Segmentation

Abstract

Joint segmentation of image sets is a challenging problem, especially when there are multiple objects with variable appearance shared among the images in the collection and the set of objects present in each particular image is itself varying and unknown. In this paper, we present a novel method to jointly segment a set of images containing objects from multiple classes. We first establish consistent functional maps across the input images, and introduce a formulation that explicitly models partial similarity across images instead of global consistency. Given the optimized maps between pairs of images, multiple groups of consistent segmentation functions are found such that they align with segmentation cues in the images, agree with the functional maps, and are mutually exclusive. The proposed fully unsupervised approach exhibits a significant improvement over the state-of-the-art methods, as shown on the cosegmentation data sets MSRC, Flickr, and PASCAL.

Optimizing the decomposition for multiple foreground cosegmentation

“…On the other hand, Wang et al [19] required the users to provide the exact number of foreground objects in input images. In practice, such user interaction or prior knowledge might not be easy to obtain, especially when the number of input images is large.…”

“…While recent research attention has been focusing on the challenging setting of MFC, most of the existing works like [10,17,18,19,20,37] choose to evaluate their performance on natural images (e.g., outdoor images with different objects presented).…”

Image Co-Segmentation and Efficiency Saliency Detection

“…Our method ismore related to Ref. 38. The main idea of our algorithm isthat the property of cliques and pixels can supplement eachother, and we iteratively optimize pixel labeling and clique potentials.…”

“…The input to our method is a pair of m × n stereo images,{IL; IR}, and a disparity map, D, which can be computed byany stereo algorithm such as the one in [38], [39]. The output ofour saliency algorithm is a pair of corresponding stereo saliencymaps {SL; SR}, where the intensity of each pixel represents theprobability of that pixel being visually important.…”

“…Ref. 38 introduced an interactive segmentationmethod using non-parametric higher-order learning algorithm.In their method, they designed two quadratic costfunctions of pixel and region likelihoods in a multi-layergraph and estimated them simultaneously. Our method ismore related to Ref.…”