Flattening Supervoxel Hierarchies by the Uniform Entropy Slice

Abstract

Supervoxel hierarchies provide a rich multiscale decomposition of a given video suitable for subsequent processing in video analysis. The hierarchies are typically computed by an unsupervised process that is susceptible to undersegmentation at coarse levels and over-segmentation at fine levels, which make it a challenge to adopt the hierarchies for later use. In this paper, we propose the first method to overcome this limitation and flatten the hierarchy into a single segmentation. Our method, called the uniform entropy slice, seeks a selection of supervoxels that balances the relative level of information in the selected supervoxels based on some post hoc feature criterion such as objectness. For example, with this criterion, in regions nearby objects, our method prefers finer supervoxels to capture the local details, but in regions away from any objects we prefer coarser supervoxels. We formulate the uniform entropy slice as a binary quadratic program and implement four different feature criteria, both unsupervised and supervised, to drive the flattening. Although we apply it only to supervoxel hierarchies in this paper, our method is generally applicable to segmentation tree hierarchies. Our experiments demonstrate both strong qualitative performance and superior quantitative performance to state of the art baselines on benchmark internet videos.

Scale-Aware Alignment of Hierarchical Image Segmentation

“…By realigning the hierarchies we aim to minimize the number of partitions from a hierarchy needed to obtain reasonable results, since we concentrate same-scale regions in the same partition. Our work also shares some similarities with [32], where they flatten supervoxel hierarchies in videos by finding a slice with uniform entropy.…”

“…However, not any labeling represents a valid slice of the tree. Following the definition in [23,32], a tree slice is a set of nodes such that every path P n , n ∈ {1, 2, ..., N } from the leaf nodev n to the root node v 0 contains one and only one node v in the slice. Figure 3(a) shows one of these paths in green.…”

“…These types of structures have been successfully used in image filtering [28], semantic segmentation [16,9], salient object detection [34], object proposals generation [22], or video segmentation [32,31].…”

Urban 3D segmentation and modelling from street view images and LiDAR point clouds

“…2 Example of rule-based segmentation of road surfaces. a 3D LiDAR point cloud segmented to road surface points (red) and other points (black); b a sketch illustrating our plane fitting to one tile ing [41]. Fast 3D-only methods exist [24], but it has also been argued that joint 2D image cues (e.g.…”

Analysis of large-scale UAV images using a multi-scale hierarchical representation

“…However, the semantic gap is rising from fine level to coarse level. To overcome the limitation of undersegmentation at coarse levels and oversegmentation at fine levels, the uniform entropy slice (UES) (Xu, Whitt, and Corso 2013) was proposed to flatten the hierarchy into a single segmentation and to seek a selection of objects that balances the objects' energy function and the relative level.…”