Unsupervised Online Video Object Segmentation With Motion Property Understanding

Abstract

Unsupervised video object segmentation aims to automatically segment moving objects over an unconstrained video without any user annotation. So far, only few unsupervised online methods have been reported in literature and their performance is still far from satisfactory, because the complementary information from future frames cannot be processed under online setting. To solve this challenging problem, in this paper, we propose a novel Unsupervised Online Video Object Segmentation (UOVOS) framework by construing the motion property to mean moving in concurrence with a generic object for segmented regions. By incorporating salient motion detection and object proposal, a pixel-wise fusion strategy is developed to effectively remove detection noise such as dynamic background and stationary objects. Furthermore, by leveraging the obtained segmentation from immediately preceding frames, a forward propagation algorithm is employed to deal with unreliable motion detection and object proposals. Experimental results on several benchmark datasets demonstrate the efficacy of the proposed method. Compared to the state-of-the-art unsupervised online segmentation algorithms, the proposed method achieves an absolute gain of 6.2%. Moreover, our method achieves better performance than the best unsupervised offline algorithm on the DAVIS-2016 benchmark dataset. Our code is available on the project website: https://github.com/visiontao/uovos.

Pop‐net: A self‐growth network for popping out the salient object in videos

“…There are 30 training and 20 validation videos. For accuracy, we use three evaluations: region similarity in terms of intersection over union (), contour accuracy (), and temporal instability () of the masks and we compare with a large set of state‐of‐the‐art methods, including the recent unsupervised techniques: UOVOS [8], LVO [5], FSEG [6], SFL [7], TIS [31], and all of the evaluation results are provided by the DAVIS [14] benchmark.…”

“…In deep learning network, appearance and motion information of the object have been used for unsupervised VOS [5][6][7][8][9][10]33,34]. [5][6][7]9] proposed a two branch network combining appearance and motion, to jointly exploit the appearance and motion feature of the object.…”

“…Jain D et al [6] proposed a two-stream FCNN that integrated motion and appearance into a unified framework, to jointly learn object segmentation and optical flow generation. In addition, Zhou T et al [8] proposed a moving object segmentation framework, by fusing motion saliency and object proposals to improve segmentation results. Xiankai L et al [10] proposed a co-attention siamese network (COSNet), which used the appearance information of the object to train the network and solved the task of unsupervised VOS from an overall perspective.…”

“…In addition, Zhou T et al. [8] proposed a moving object segmentation framework, by fusing motion saliency and object proposals to improve segmentation results. Xiankai L et al.…”

Weakly Supervised Video Object Segmentation

“…To segment moving objects, [55] trains a network to directly output the segmentation from motion, which is then augmented by an appearance channel in [21]. Further, [72] improves the segmentation by incorporating salient motion detection with object proposals. With region augmentation and reduction, [26] segments video objects based on the recurrence property of primary object.…”