Occlusion-Aware Method for Temporally Consistent Superpixels

Abstract

A wide variety of computer vision applications rely on superpixel or supervoxel algorithms as a preprocessing step. This underlines the overall importance that these approaches have gained in recent years. However, most methods show a lack of temporal consistency or fail in producing temporally stable superpixels. In this paper, we present an approach to generate temporally consistent superpixels for video content. Our method is formulated as a contour-evolving expectation-maximization framework, which utilizes an efficient label propagation scheme to encourage the preservation of superpixel shapes and their relative positioning over time. By explicitly detecting the occlusion of superpixels and the disocclusion of new image regions, our framework is able to terminate and create superpixels whose corresponding image region becomes hidden or newly appears. Additionally, the occluded parts of superpixels are incorporated in the further optimization. This increases the compliance of the superpixel flow with the optical flow present in the scene. Using established benchmark suites, we show the performance of our approach in comparison to state-of-the-art supervoxel and superpixel algorithms for video content.

Feature-Aware Uniform Tessellations on Video Manifold for Content-Sensitive Supervoxels

“…However, even stateof-the-art optical flow estimation methods [36] are still imperfect and may introduce extra errors into supervoxel computation. Recently, Reso et al [31] propose a novel formulation specifically designed for handling occlusions.…”

“…Many methods have been proposed for computing supervoxels, including energy minimization by graph cut [38], non-parametric feature-space analysis [28], graphbased merging [9], [13], [42], contour-evolving optimization [17], [21], [31], optimization of normalized cuts [33], [7], generative probabilistic framework [5] and hybrid clustering [30], [43], etc. These methods can be classified according • R. Yi to different representation formats: (1) temporal superpixels [5], [4], [17], [21], [30], [31], [39]: supervoxels are represented in each frame and their labels are temporally consistent in adjacent frames, and (2) supervoxels [7], [9], [13], [28], [33], [38], [42], [43]: they are 3D primitive volumes whose union forms the video volume. Note that these two representations can be transferred to each other.…”

SMBFT: A Modified Fuzzy c -Means Algorithm for Superpixel Generation

“…Recently, superpixel segmentation methods are becoming more and more popular. These methods can be mainly divided into two categories: graph-based methods and gradient ascent methods [5][6][7][8][9][10][11][12].…”