RANet: Ranking Attention Network for Fast Video Object Segmentation

Abstract

Despite online learning (OL) techniques have boosted the performance of semi-supervised video object segmentation (VOS) methods, the huge time costs of OL greatly restrict their practicality. Matching based and propagation based methods run at a faster speed by avoiding OL techniques. However, they are limited by sub-optimal accuracy, due to mismatching and drifting problems. In this paper, we develop a real-time yet very accurate Ranking Attention Network (RANet) for VOS. Specifically, to integrate the insights of matching based and propagation based methods, we employ an encoder-decoder framework to learn pixellevel similarity and segmentation in an end-to-end manner. To better utilize the similarity maps, we propose a novel ranking attention module, which automatically ranks and selects these maps for fine-grained VOS performance. Experiments on DAVIS 16 and DAVIS 17 datasets show that our RANet achieves the best speed-accuracy trade-off, e.g., with 33 milliseconds per frame and J &F=85.5% on DAVIS 16 . With OL, our RANet reaches J &F=87.1% on DAVIS 16

EGNet: Edge Guidance Network for Salient Object Detection

“…It has received widespread attention recently and been widely used in many vision and image processing related areas, such as contentaware image editing [6], object recognition [42], photosynth [4], non-photo-realist rendering [41], weakly supervised semantic segmentation [19] and image retrieval [15]. Besides, there are many works focusing on video salient object detection [12,54] and RGB-D salient object detection [11,66].…”

Learning What to Learn for Video Object Segmentation

“…However, this strategy easily leads to overfitting to the initial target appearance and impractically long run-times. More recent methods [34,13,32,23,36,24,17] therefore integrate target-specific appearance models into the segmentation architecture. In addition to improved run-times, many of these methods can also benefit from full end-to-end learning, which has been shown to have a crucial impact on performance [32,14,24].…”

“…This information is then sent to the segmentation network to predict the target mask. The method [36] predicts template correlation filters given the input target mask. Target classification is then performed by applying the correlation filters on the the test frame.…”

Pyramid Constrained Self-Attention Network for Fast Video Salient Object Detection

“…Video Object Segmentation (VOS) (Perazzi et al 2016) is related to VSOD, and it mainly includes Unsupervised VOS (UVOS) (Song et al 2018) and semi-supervised VOS (Wang et al 2019b). Semi-supervised VOS aims to segment specific objects which are assigned by the first frame, while UVOS predicts masks for primary objects in a video, with no other hints such as reference masks.…”