Fast Video Object Segmentation Using the Global Context Module

Yu, Lie; Shen, Zhuoran; Shan, Ying

doi:10.1007/978-3-030-58607-2_43

Cited by 89 publications

(67 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We use ADAM optimizer for training our model. Firstly, we pre-train our model with synthetic video clips from image dataset, after then we train it with video dataset with single GPU following [9], [11]- [14].…”

Section: Methodsmentioning

confidence: 99%

“…Pre-Train With Images: We follow [10], [13], [14] pre-training method, which applies a random affine transformation to a static image for generating synthetic video clips. We use the saliency detection dataset MSRA10K [41], ECSSD [42], and HKU-IS [43] for various static images.…”

Section: Methodsmentioning

confidence: 99%

“…because the amount of memory is increased over time and the size of memory is the square of the resolution of an input feature map. To lower this huge complexity, GC [14] does not stack memory at each time frame, but accumulates them into one, which has smaller size than an unit memory of STM.…”

Section: B Vos Based On Template Matchingmentioning

confidence: 99%

“…This strategy makes the model's parameters more specialized in each target object, but it requires additional time and memory for fine-tuning at the inference stage. Other approaches, matching methods [9]- [11], [13], [14], construct target information as a template and match it to the current frames for finding the target. This method does not need extra-training in inference stage, but it is not fast enough for real-world applications.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Adaptive Template and Transition Map for Real-Time Video Object Segmentation

et al. 2021

View full text Add to dashboard Cite

Semi-supervised video object segmentation (semi-VOS) is required for many visual applications. This task is tracking class-agnostic objects from a given segmentation mask. Various approaches have been developed and achieved high accuracy in this field, but these previous models are hard to be utilized in real-world applications due to slow inference time and tremendous complexity. To significantly speed up inference while reducing performance gaps from those previous models, we introduce a fast segmentation model based on a template matching method and auxiliary loss with a transition map. Our template matching method consists of short-term and long-term matching. The short-term matching enhances target object localization by focusing on neighboring frames, while long-term matching improves fine details and handles object shape-changing by considering long-range frames. However, since both matching processes generate each template based on the previously estimated masks, this incurs error propagation for tracking objects in the next frames. To mitigate this problem, we add auxiliary loss with a newly proposed transition map for encouraging correction power to create accurate masks of the target object. Our model obtains 81.1% J &F score at the speed of 78.3 FPS on the DAVIS16 benchmark and achieves 1.4× faster speed and 11.3% higher accuracy than SiamMask, one of the fast semi-VOS models.INDEX TERMS Semi-supervised video object segmentation, video object segmentation, video object tracking, deep learning.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: B Vos Based On Template Matchingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Adaptive Template and Transition Map for Real-Time Video Object Segmentation

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Memory networks have been introduced to enhance the reasoning ability of the model in VideoQA [8,9] and video object segmentation [10,11,12], but have never been introduced in video semantic segmentation as we know. [8] uses episodic memory to conduct multiple cycles of inference by interacting the question with video features conditioned on current memory.…”

Section: Introductionmentioning

confidence: 99%

Temporal Memory Attention for Video Semantic Segmentation

Wang

Liu

2021

2021 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Video semantic segmentation requires to utilize the complex temporal relations between frames of the video sequence. Previous works usually exploit accurate optical flow to leverage the temporal relations, which suffer much from heavy computational cost. In this paper, we propose a Temporal Memory Attention Network (TMANet) to adaptively integrate the long-range temporal relations over the video sequence based on the self-attention mechanism without exhaustive optical flow prediction. Specially, we construct a memory using several past frames to store the temporal information of the current frame. We then propose a temporal memory attention module to capture the relation between the current frame and the memory to enhance the representation of the current frame. Our method achieves new state-of-theart performances on two challenging video semantic segmentation datasets, particularly 80.3% mIoU on Cityscapes and 76.5% mIoU on CamVid with ResNet-50.

show abstract

XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Model

Cheng

Schwing

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Fast Video Object Segmentation Using the Global Context Module

Cited by 89 publications

References 37 publications

Adaptive Template and Transition Map for Real-Time Video Object Segmentation

Adaptive Template and Transition Map for Real-Time Video Object Segmentation

Temporal Memory Attention for Video Semantic Segmentation

XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Model

Contact Info

Product

Resources

About