Discontinuity-aware video object cutout

Abstract

Existing video object cutout systems can only deal with limited cases. They usually require detailed user interactions to segment real-life videos, which often suffer from both inseparable statistics (similar appearance between foreground and background) and temporal discontinuities (e.g. large movements, newly-exposed regions following disocclusion or topology change).In this paper, we present an efficient video cutout system to meet this challenge. A novel directional classifier is proposed to handle temporal discontinuities robustly, and then multiple classifiers are incorporated to cover a variety of cases. The outputs of these classifiers are integrated via another classifier, which is learnt from real examples. The foreground matte is solved by a coherent matting procedure, and remaining errors can be removed easily by additive spatio-temporal local editing. Experiments demonstrate that our system performs more robustly and more intelligently than existing systems in dealing with various input types, thus saving a lot of user labor and time.

Selective Video Object Cutout

“…Typical interactive methods propagate the given annotations to the entire video sequence, by tracking them using spatiotemporal Markov random fields based probabilistic models [6], [31], [32], [33], [13], or frame-matching based propagation [1], [3], [7], [34], and employing various features such as color, shape, and motion.…”

“…Finally, morphological 3 The indices (lt, rt) might not exist if t < n 1 or t > n K . For clarity we omit such cases, since they do not affect the method description.…”

High-Quality Depth Recovery via Interactive Multi-view Stereo

“…Several interactive video segmentation/matting techniques [2,30,3,21,35] have also been developed. Early methods (e.g.…”

“…Price et al [21] also proposed to combine kinds of cues to construct a similar video object segmentation framework. Zhong et al [35] presented a bidirectional propagation strategy and combined different classifiers based on a learning-based method. However, these methods are generally designed for handling moving objects, and are not specifically optimized for handling static objects and depth repairing.…”

“…By considering and coping with occlusion, the segmentation result is successfully propagated over dozens of frames. We have compared our method with the most recent state-of-the-art work [3,35]. Our propagation method is more robust especially when there is strong occlusion.…”