Scale invariant cosegmentation for image groups

Abstract

Our primary interest is in generalizing the problem of Cosegmentation to a large group of images, that is, concurrent segmentation of common foreground region(s) from multiple images. We further wish for our algorithm to offer scale invariance (foregrounds may have arbitrary sizes in different images) and the running time to increase (no more than) near linearly in the number of images in the set. What makes this setting particularly challenging is that even if we ignore the scale invariance desiderata, the Cosegmentation problem, as formalized in many recent papers (except [1]), is already hard to solve optimally in the two image case. A straightforward extension of such models to multiple images leads to loose relaxations; and unless we impose a distributional assumption on the appearance model, existing mechanisms for image-pair-wise measurement of foreground appearance variations lead to significantly large problem sizes (even for moderate number of images). This paper presents a surprisingly easy to implement algorithm which performs well, and satisfies all requirements listed above (scale invariance, low computational requirements, and viability for the multiple image setting). We present qualitative and technical analysis of the properties of this framework.

Semantic Object Selection

“…9 of the 14 classes of MSRC are present in PASCAL VOC 2010, we thus compute results on 9 classes of MSRC (around 30 images per class). We compare our performance with [10,19,9,8] as reported in [8]. We also compare against the recent Object Discovery work [21] which uses dense correspondences between images to capture the visual variability of common object.…”

“…Cosegmentation methods [9,8,10,19] operate on multiple input images and select in each image a common object. Such methods could be adapted to our problem by performing an Internet search on the query object and performing cosegmentation on the results together with the query image.…”

Unsupervised Multi-class Joint Image Segmentation

“…This is a standard binary segmentation setting, therefore, many existing single-class co-segmentation algorithms are applicable. Table 1 gives a quantitative comparison with [7,8,11], and the same classes are selected as reported in [7]. [7] is designed for multi-class segmentation and [8] and [11] are state-of-the-art foreground-background cosegmentation methods.…”

“…Later on, other kinds of features were also utilized to exploit the relationship between image foregrounds, such as SIFT [11], saliency [1], and Gabor features [5]. To address the cosegmentation of multiple images, Joulin et al formulated the co-segmentation task as a discriminative clustering problem by clustering the image pixels into foreground and background [6].…”

Tell Me What You See and I Will Show You Where It Is

“…Most previous methods focus on the setting where a single foreground object is present in all images [33,16]. This setting has been extended to segment multiple objects by analyzing the subspace structure of multiple foreground objects [17], using a greedy procedure with submodular optimization [11], or by grouping image regions via spectral discriminative clustering [10].…”