PReMVOS: Proposal-Generation, Refinement and Merging for Video Object Segmentation

Luiten, Jonathon; Voigtlaender, Paul; Leibe, Bastian

doi:10.1007/978-3-030-20870-7_35

Cited by 230 publications

(281 citation statements)

References 31 publications

Supporting

Mentioning

272

Contrasting

Unclassified

Order By: Relevance

“…Using better detections and better segmentations does benefit our method. Using [20] segmentations instead of [38], sMOTSA increases from 78.2 to 82.8 for cars and from 50.1 to 59.4 for pedestrians (4.6 and 9.3 percentage points). Using [29] detections instead of [38], sMOTSA increases another 2.9 percentage points from 82.8 to 85.7 for cars.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Track to Reconstruct and Reconstruct to Track

Luiten

Fischer

Leibe

2020

IEEE Robot. Autom. Lett.

Self Cite

112

View full text Add to dashboard Cite

Object tracking and 3D reconstruction are often performed together, with tracking used as input for reconstruction. However, the obtained reconstructions also provide useful information for improving tracking. We propose a novel method that closes this loop, first tracking to reconstruct, and then reconstructing to track. Our approach, MOTSFusion (Multi-Object Tracking, Segmentation and dynamic object Fusion), exploits the 3D motion extracted from dynamic object reconstructions to track objects through long periods of complete occlusion and to recover missing detections. Our approach first builds up short tracklets using 2D optical flow, and then fuses these into dynamic 3D object reconstructions. The precise 3D object motion of these reconstructions is used to merge tracklets through occlusion into long-term tracks, and to locate objects when detections are missing. On KITTI, our reconstructionbased tracking reduces the number of ID switches of the initial tracklets by more than 50%, and outperforms all previous approaches for both bounding box and segmentation tracking.

show abstract

Section: Methodsmentioning

confidence: 99%

“…We initially estimate a segmentation mask for each bounding box detection. We use a fully convolutional neural network from [20] which we call BB2SegNet. This crops and resizes an image region given by a bounding box to a 385 × 385 patch and outputs a segmentation mask for each box.…”

Section: Our Approachmentioning

confidence: 99%

Track to Reconstruct and Reconstruct to Track

Luiten

Fischer

Leibe

2020

IEEE Robot. Autom. Lett.

Self Cite

112

View full text Add to dashboard Cite

show abstract

“…Through combining an instance bounding proposals and coarse masks, we obtain the instancelevel mask for each primary object. Finally, to connect multiple instances across different frames, we use overlap ratio and optical flow as an association metric [38] to match different instance-level masks.…”

Section: Experimental Setup Datasets and Metricsmentioning

confidence: 99%

Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks

Yang

Shen

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

266

133

View full text Add to dashboard Cite

This work proposes a novel attentive graph neural network (AGNN) for zero-shot video object segmentation (ZVOS). The suggested AGNN recasts this task as a process of iterative information fusion over video graphs. Specifically, AGNN builds a fully connected graph to efficiently represent frames as nodes, and relations between arbitrary frame pairs as edges. The underlying pair-wise relations are described by a differentiable attention mechanism. Through parametric message passing, AGNN is able to efficiently capture and mine much richer and higher-order relations between video frames, thus enabling a more complete understanding of video content and more accurate foreground estimation. Experimental results on three video segmentation datasets show that AGNN sets a new state-ofthe-art in each case. To further demonstrate the generalizability of our framework, we extend AGNN to an additional task: image object co-segmentation (IOCS). We perform experiments on two famous IOCS datasets and observe again the superiority of our AGNN model. The extensive experiments verify that AGNN is able to learn the underlying semantic/appearance relationships among video frames or related images, and discover the common objects.

show abstract

“…Semi-supervised VOS methods are provided with a pixelwise mask identifying the target object in the first frame of a video. When aiming at very high segmentation accuracy, methods generally perform online fine-tuning on the basis of this supervision [3,25,35,40,43,50,62], sometimes exploiting data-augmentation techniques [3,25] or self-supervision [62]. As online fine-tuning can take up to several minutes per video, many recently proposed methods renounce to it and instead aim at a faster online speed (e.g., [7,8,64]).…”

Section: Related Workmentioning

confidence: 99%

“…PReMVOS [40] 84.9 88.6 -OSVOS [3] 79.8 80.6 -MSK [50] 79.7 75.4 -PML [7] 75.5 79.3 -SFL [9] 76.1 76.0 -VPN [52] 70. 2 pruning.…”

Section: Comparison With the State Of The Artmentioning

confidence: 99%

Anchor Diffusion for Unsupervised Video Object Segmentation

Zhao

Wang

Bertinetto³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

115

View full text Add to dashboard Cite

Unsupervised video object segmentation has often been tackled by methods based on recurrent neural networks and optical flow. Despite their complexity, these kinds of approaches tend to favour short-term temporal dependencies and are thus prone to accumulating inaccuracies, which cause drift over time. Moreover, simple (static) image segmentation models, alone, can perform competitively against these methods, which further suggests that the way temporal dependencies are modelled should be reconsidered. Motivated by these observations, in this paper we explore simple yet effective strategies to model long-term temporal dependencies. Inspired by the non-local operators of [70], we introduce a technique to establish dense correspondences between pixel embeddings of a reference "anchor" frame and the current one. This allows the learning of pairwise dependencies at arbitrarily long distances without conditioning on intermediate frames. Without online supervision, our approach can suppress the background and precisely segment the foreground object even in challenging scenarios, while maintaining consistent performance over time. With a mean IoU of 81.7%, our method ranks first on the DAVIS-2016 leaderboard of unsupervised methods, while still being competitive against state-of-the-art online semisupervised approaches. We further evaluate our method on the FBMS dataset and the ViSal video saliency dataset, showing results competitive with the state of the art.

show abstract

PReMVOS: Proposal-Generation, Refinement and Merging for Video Object Segmentation

Cited by 230 publications

References 31 publications

Track to Reconstruct and Reconstruct to Track

Track to Reconstruct and Reconstruct to Track

Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks

Anchor Diffusion for Unsupervised Video Object Segmentation

Contact Info

Product

Resources

About