Learning Video Object Segmentation from Static Images

Perazzi, Federico; Khoreva, Anna; Benenson, Rodrigo; Schiele, Bernt; Sorkine-Hornung, Alexander

doi:10.1109/cvpr.2017.372

Cited by 513 publications

(522 citation statements)

References 39 publications

Supporting

Mentioning

520

Contrasting

Order By: Relevance

“…As shown in Table 2, we observe that the STCNN method produces the best results with 0.796 mean IoU, which surpasses the state-of-the-art results, i.e., MR-FCNN [2] (0.784 mean IoU), with 0.012 mIoU. Compared to the optical flow based methods [46,38], our STCNN method performs well on fast moving objects, such as car and cat. The estimation of optical flow for fast moving objets is inaccurate, affecting the segmentation accuracy.…”

Section: Youtube-objects Datasetmentioning

confidence: 78%

“…As shown in Table 1, our algorithm outperforms the existing semi-supervised algorithms (e.g., OSVOS [3] and MSK [38]) and unsupervised algorithms (e.g., ARP [28] and LVO [45]) with 0.838 mean region similarity J , 0.838 mean contour accuracy F, and 0.191 temporal stability T , except CRN [15] and OnAVOS [47]. The OnAVOS algorithm [47] updates the network online using training examples selected based on the confidence of the network and the spatial configuration, which requires heavy consumption of time and computation resource.…”

Section: Comparison With State-of-the-artsmentioning

confidence: 82%

“…We compare the proposed STCNN method to 11 stateof-the-art semi-supervised algorithms, namely BVS [34], JFS [35], SCF [20], MRFCNN [2], LT [25], OSVOS [3], MSK [38], OFL [46], CRN [15], DRL [12], and OnAVOS [47] in Table 2. As shown in Table 2, we observe that the STCNN method produces the best results with 0.796 mean IoU, which surpasses the state-of-the-art results, i.e., MR-FCNN [2] (0.784 mean IoU), with 0.012 mIoU.…”

Section: Youtube-objects Datasetmentioning

confidence: 99%

“…The symbol ↑ means higher scores indicate better performance. Bold font indicates the best result.Method BVS[34] JFS[35] SCF[20] MRFCNN[2] LT[25] OSVOS[3] MSK[38] OFL[46] CRN[15] DRL[12] OnAVOS[47] Ours…”

mentioning

confidence: 99%

See 3 more Smart Citations

Spatiotemporal CNN for Video Object Segmentation

Xu¹,

Wen²,

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

In this paper, we present a unified, end-to-end trainable spatiotemporal CNN model for VOS, which consists of two branches, i.e., the temporal coherence branch and the spatial segmentation branch. Specifically, the temporal coherence branch pretrained in an adversarial fashion from unlabeled video data, is designed to capture the dynamic appearance and motion cues of video sequences to guide object segmentation. The spatial segmentation branch focuses on segmenting objects accurately based on the learned appearance and motion cues. To obtain accurate segmentation results, we design a coarse-to-fine process to sequentially apply a designed attention module on multi-scale feature maps, and concatenate them to produce the final prediction. In this way, the spatial segmentation branch is enforced to gradually concentrate on object regions. These two branches are jointly fine-tuned on video segmentation sequences in an end-to-end manner. Several experiments are carried out on three challenging datasets (i.e., DAVIS-2016, DAVIS-2017 and Youtube-Object) to show that our method achieves favorable performance against the stateof-the-arts. Code is available at https://github. com/longyin880815/STCNN .

show abstract

Section: Youtube-objects Datasetmentioning

confidence: 78%

Section: Comparison With State-of-the-artsmentioning

confidence: 82%

Section: Youtube-objects Datasetmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Spatiotemporal CNN for Video Object Segmentation

Xu¹,

Wen²,

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

show abstract

“…Video segmentation problems involve several subproblems which can be classified as semantic segmentation [29] and general object segmentation [30,31]. For video object segmentation problems, convolutional neural network (CNN)-based approaches have made great advances in both unsupervised learning [30,[32][33][34][35] and semi-supervised learning [36][37][38]. CNNs have recently been integrated with gPb-UCM.…”

Section: Deep Neural Networkmentioning

confidence: 99%

Supervoxel-based segmentation of 3D imagery with optical flow integration for spatiotemporal processing

Huang

Yang

Ranka

et al. 2018

IPSJ T Comput Vis Appl

View full text Add to dashboard Cite

The past 20 years has seen a progressive evolution of computer vision algorithms for unsupervised 2D image segmentation. While earlier efforts relied on Markov random fields and efficient optimization (graph cuts, etc.), the next wave of methods beginning in the early part of this century were, in the main, stovepiped. Of these 2D segmentation efforts, one of the most popular and, indeed, one that comes close to being a state of the art method is the ultrametric contour map (UCM). The pipelined methodology consists of (i) computing local, oriented responses, (ii) graph creation, (iii) eigenvector computation (globalization), (iv) integration of local and global information, (v) contour extraction, and (vi) superpixel hierarchy construction. UCM performs well on a range of 2D tasks. Consequently, it is somewhat surprising that no 3D version of UCM exists at the present time. To address that lack, we present a novel 3D supervoxel segmentation method, dubbed 3D UCM, which closely follows its 2D counterpart while adding 3D relevant features. The methodology, driven by supervoxel extraction, combines local and global gradient-based features together to first produce a low-level supervoxel graph. Subsequently, an agglomerative approach is used to group supervoxel structures into a segmentation hierarchy with explicitly imposed containment of lower-level supervoxels in higher-level supervoxels. Comparisons are conducted against state of the art 3D segmentation algorithms. The considered applications are 3D spatial and 2D spatiotemporal segmentation scenarios. For the latter comparisons, we present results of 3D UCM with and without optical flow video pre-processing. As expected, when motion correction beyond a certain range is required, we demonstrate that 3D UCM in conjunction with optical flow is a very useful addition to the pantheon of video segmentation methods.

show abstract