Flow Adaptive Video Object Segmentation

Lin, Fanqing; Chou, Yao; Martinez, Tony

doi:10.1016/j.imavis.2019.103864

Cited by 18 publications

(10 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In many works, optical flow is used to convey additional information to guide the segmentation network for refinement [6], [23]. Similarly, FAVOS [18] leverages optical flow to warp previous predictions and uses them with current predictions to generate accurate ground truth masks for online learning. Segflow [21] designs end-to-end networks with two sub-networks complementing each other for image segmentation and optical flow, respectively, to estimate target masks.…”

Section: Optical Flow For Temporal Consistencymentioning

confidence: 99%

“…Optical flow is one of the popular methods to resolve the mentioned problems for diverse video applications by correctly estimating pixel-wise movement vectors or trajectories. In semi-VOS tasks, it propagates a given mask or features across frames to re-align the information for current frames [18]- [21]. However, even though the optical flow is only a part of the entire process, it needs huge resource to provide too much information for the segmentation tasks.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Adaptive Template and Transition Map for Real-Time Video Object Segmentation

et al. 2021

View full text Add to dashboard Cite

Semi-supervised video object segmentation (semi-VOS) is required for many visual applications. This task is tracking class-agnostic objects from a given segmentation mask. Various approaches have been developed and achieved high accuracy in this field, but these previous models are hard to be utilized in real-world applications due to slow inference time and tremendous complexity. To significantly speed up inference while reducing performance gaps from those previous models, we introduce a fast segmentation model based on a template matching method and auxiliary loss with a transition map. Our template matching method consists of short-term and long-term matching. The short-term matching enhances target object localization by focusing on neighboring frames, while long-term matching improves fine details and handles object shape-changing by considering long-range frames. However, since both matching processes generate each template based on the previously estimated masks, this incurs error propagation for tracking objects in the next frames. To mitigate this problem, we add auxiliary loss with a newly proposed transition map for encouraging correction power to create accurate masks of the target object. Our model obtains 81.1% J &F score at the speed of 78.3 FPS on the DAVIS16 benchmark and achieves 1.4× faster speed and 11.3% higher accuracy than SiamMask, one of the fast semi-VOS models.INDEX TERMS Semi-supervised video object segmentation, video object segmentation, video object tracking, deep learning.

show abstract

Section: Optical Flow For Temporal Consistencymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Adaptive Template and Transition Map for Real-Time Video Object Segmentation

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Segflow [2] designs two branches of image segmentation and optical flow, and bidirectionally combines both information into a unified framework to estimate target masks. Similarly, FAVOS [14] and CRN [5] utilize optical flow information to refine a coarse segmentation mask into an accurate mask.…”

Section: Related Workmentioning

confidence: 99%

“…A variety of methods have been proposed for solving video object segmentation including online learning [27,16], mask propagation [14,2], and template matching [36,19]. A common theme across most of these previous methods is to use information from previous frames -either just the first frame, some of the previous frames (first and last being a popular option) or all the previous frames -to pro-duce high quality segmentation mask.…”

Section: Introductionmentioning

confidence: 99%

Learning Dynamic Network Using a Reuse Gate Function in Semi-supervised Video Object Segmentation

Park¹,

Yoo²,

Jeong³

et al. 2020

Preprint

View full text Add to dashboard Cite

Current state-of-the-art approaches for Semi-supervised Video Object Segmentation (Semi-VOS) propagates information from previous frames to generate segmentation mask for the current frame. This results in high-quality segmentation across challenging scenarios such as changes in appearance and occlusion. But it also leads to unnecessary computations for stationary or slow-moving objects where the change across frames is minimal. In this work, we exploit this observation by using temporal information to quickly identify frames with minimal change and skip the heavyweight mask generation step. To realize this efficiency, we propose a novel dynamic network that estimates change across frames and decides which path -computing a full network or reusing previous frame's feature -to choose depending on the expected similarity. Experimental results show that our approach significantly improves inference speed without much accuracy degradation on challenging Semi-VOS datasets -DAVIS 16, DAVIS 17, and YouTube-VOS. Furthermore, our approach can be applied to multiple Semi-VOS methods demonstrating its generality.

show abstract

“…As illustrated in Fig. 1 (b), several recent methods opt for the simplex strategy [39,61,65,74,80,100,141], which is either appearance-based or motion-guided. Although these two strategies have achieved promising results, they both fail to consider the mutual restraint between the appearance and motion features that both guide human visual attention allocation during dynamic observation, according to previous studies in cognitive psychology [50,99,119] and computer vision [44,107].…”

Section: Introductionmentioning

confidence: 99%

Full-Duplex Strategy for Video Object Segmentation

et al. 2021

Preprint

View full text Add to dashboard Cite

Previous video object segmentation approaches mainly focus on using simplex solutions between appearance and motion, limiting feature collaboration efficiency among and across these two cues. In this work, we study a novel and efficient full-duplex strategy network (FSNet) to address this issue, by considering a better mutual restraint scheme between motion and appearance in exploiting the crossmodal features from the fusion and decoding stage. Specifically, we introduce the relational cross-attention module (RCAM) to achieve bidirectional message propagation across embedding sub-spaces. To improve the model's robustness and update the inconsistent features from the spatial-temporal embeddings, we adopt the bidirectional purification module (BPM) after the RCAM. Extensive experiments on five popular benchmarks show that our FSNet is robust to various challenging scenarios (e.g., motion blur, occlusion) and achieves favourable performance against existing cutting-edges both in the video object segmentation and video salient object detection tasks. The project is publicly available at: https://dpfan.net/FSNet.

show abstract

Flow Adaptive Video Object Segmentation

Cited by 18 publications

References 39 publications

Adaptive Template and Transition Map for Real-Time Video Object Segmentation

Adaptive Template and Transition Map for Real-Time Video Object Segmentation

Learning Dynamic Network Using a Reuse Gate Function in Semi-supervised Video Object Segmentation

Full-Duplex Strategy for Video Object Segmentation

Contact Info

Product

Resources

About