Rethinking Unsupervised Neural Superpixel Segmentation

Abstract

Recently, the concept of unsupervised learning for superpixel segmentation via CNNs has been studied. Essentially, such methods generate superpixels by convolutional neural network (CNN) employed on a single image, and such CNNs are trained without any labels or further information. Thus, such approach relies on the incorporation of priors, typically by designing an objective function that guides the solution towards a meaningful superpixel segmentation. In this paper we propose three key elements to improve the efficacy of such networks: (i) the similarity of the soft superpixelated image compared to the input image, (ii) the enhancement and consideration of object edges and boundaries and (iii) a modified architecture based on atrous convolution, which allow for a wider field of view, functioning as a multi-scale component in our network. By experimenting with the BSDS500 dataset, we find evidence to the significance of our proposal, both qualitatively and quantitatively.

Unsupervised Image Semantic Segmentation Through Superpixels and Graph Neural Networks

“…Classical methods like SLIC [1] use Euclidean coordinates and color space similarity to define superpixels, while other works like SEEDS [52] and FH [17] define an energy functional that is minimized by graph cuts. Recently, methods like [30] proposed to use CNNs to extract superpixels in a supervised manner, and following that it was shown in [51,61,16] that a CNN is also beneficial for unsupervised superpixel extraction and to substantially improve the accuracy of classical methods like SLIC, SEEDS and FH. We note that in addition to performing better than classical methods, the mentioned CNN based models are fullydifferentiable, which is a desired property that we leverage in this work.…”

“…For example, by demanding signal reconstruction and enforcing MIM between inputs and their reconstruction or predictions. This concept was found to be useful in a wide array of applications, from image superpixel segmentation [51,61,16] to unsupervised image semantic segmentation [31,42,41] to unsupervised graph related tasks [54]. In this paper we utilize mutual information maximization in a similar fashion to [42] by demanding similar prediction given different rotations and rasterization of the learnt convolution kernels.…”

“…Our superpixel extraction neural network (SPNN) is based on the proposed methods in [51,61,16] which were shown to significantly improve the standard metrics of boundary precision and achievable segmentation accuracy compared to methods like SLIC and SEEDS. The core idea is to treat the superpixel segmentation as a classification problem using a CNN.…”

“…We also follow [16] and require the similarity of the predicted soft-superpixelated image ÎP and the input image I. Therefore our total reconstruction loss is given by…”

“…Lastly, and similar to [61,16], we also consider correspondence between the edges of the input image I, the reconstructed image Ĩ and the soft-superpixelated image ÎP defined in Eq. ( 2).…”