Online Video Object Segmentation via Convolutional Trident Network

Jang, Won-Dong

doi:10.1109/cvpr.2017.790

Cited by 113 publications

(67 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It has created a large number of synthetic video training data from Pascal VOC [11,12], ECSSD [49] and MSRA10K [7] DAVIS 2017 benchmark, we exclude PReMVOS [38] and OSVOS+ [39] as they both use multiple specialized networks in multiple processes to refine their results. For DAVIS 2016, we compare with OnAVOS [52], FAVOS [5], OSVOS [3], MSK [42], PML [4], SFL [6], OSMN [57], CTN [27] and VPN [26]. We detect multiple objects and evaluate in the way for single-object.…”

Section: Compare With Other Methodsmentioning

confidence: 99%

LIP: Learning Instance Propagation for Video Object Segmentation

Lyu

Vosselman

Xia

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

In recent years, the task of segmenting foreground objects from background in a video, i.e. video object segmentation (VOS), has received considerable attention. In this paper, we propose a single end-to-end trainable deep neural network, convolutional gated recurrent Mask-RCNN, for tackling the semi-supervised VOS task. We take advantage of both the instance segmentation network (Mask-RCNN) and the visual memory module (Conv-GRU) to tackle the VOS task. The instance segmentation network predicts masks for instances, while the visual memory module learns to selectively propagate information for multiple instances simultaneously, which handles the appearance change, the variation of scale and pose and the occlusions between objects. After offline and online training under purely instance segmentation losses, our approach is able to achieve satisfactory results without any post-processing or synthetic video data augmentation. Experimental results on DAVIS 2016 dataset and DAVIS 2017 dataset have demonstrated the effectiveness of our method for video object segmentation task.

show abstract

Section: Compare With Other Methodsmentioning

confidence: 99%

LIP: Learning Instance Propagation for Video Object Segmentation

Lyu

Vosselman

Xia

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

show abstract

“…Despite reduced accuracy, several approaches avoid invoking expensive fine-tuning procedures in the first frame. Some methods rely on optical flow coupled with refinement [15,29]. Li et al proposed DyeNet [18], which combines optical flow with an object proposal network, interleaving bidirectional mask-propagation and target re-identification.…”

Section: Related Workmentioning

confidence: 99%

A Generative Appearance Model for End-To-End Video Object Segmentation

Johnander

Danelljan

Brissman

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

174

126

View full text Add to dashboard Cite

One of the fundamental challenges in video object segmentation is to find an effective representation of the target and background appearance. The best performing approaches resort to extensive fine-tuning of a convolutional neural network for this purpose. Besides being prohibitively expensive, this strategy cannot be truly trained end-to-end since the online fine-tuning procedure is not integrated into the offline training of the network.To address these issues, we propose a network architecture that learns a powerful representation of the target and background appearance in a single forward pass. The introduced appearance module learns a probabilistic generative model of target and background feature distributions. Given a new image, it predicts the posterior class probabilities, providing a highly discriminative cue, which is processed in later network modules. Both the learning and prediction stages of our appearance module are fully differentiable, enabling true end-to-end training of the entire segmentation pipeline. Comprehensive experiments demonstrate the effectiveness of the proposed approach on three video object segmentation benchmarks. We close the gap to approaches based on online fine-tuning on DAVIS17, while operating at 15 FPS on a single GPU. Furthermore, our method outperforms all published approaches on the largescale YouTube-VOS dataset.

show abstract

“…Subsequent works [2] use optical flow for temporal consistency after using Markov random fields based on features taken from a Convolutional Neural Network. An alternative to gain temporal coherence is to use the predicted masks in the previous frames as guidance for next frames [7,11,20,34]. In the same direction, [10] propagate information forward by using spatio-temporal features.…”

Section: Related Workmentioning

confidence: 99%

RVOS: End-To-End Recurrent Network for Video Object Segmentation

Ventura

Bellver

Girbau

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

221

117

View full text Add to dashboard Cite

Multiple object video object segmentation is a challenging task, specially for the zero-shot case, when no object mask is given at the initial frame and the model has to find the objects to be segmented along the sequence. In our work, we propose a Recurrent network for multiple object Video Object Segmentation (RVOS) that is fully end-to-end trainable. Our model incorporates recurrence on two different domains: (i) the spatial, which allows to discover the different object instances within a frame, and (ii) the temporal, which allows to keep the coherence of the segmented objects along time. We train RVOS for zero-shot video object segmentation and are the first ones to report quantitative results for DAVIS-2017 and YouTube-VOS benchmarks. Further, we adapt RVOS for one-shot video object segmentation by using the masks obtained in previous time steps as inputs to be processed by the recurrent module. Our model reaches comparable results to state-of-the-art techniques in YouTube-VOS benchmark and outperforms all previous video object segmentation methods not using online learning in the DAVIS-2017 benchmark. Moreover, our model achieves faster inference runtimes than previous methods, reaching 44ms/frame on a P100 GPU.

show abstract

Online Video Object Segmentation via Convolutional Trident Network

Cited by 113 publications

References 43 publications

LIP: Learning Instance Propagation for Video Object Segmentation

LIP: Learning Instance Propagation for Video Object Segmentation

A Generative Appearance Model for End-To-End Video Object Segmentation

RVOS: End-To-End Recurrent Network for Video Object Segmentation

Contact Info

Product

Resources

About