Video Object Segmentation without Temporal Information

Abstract

Video Object Segmentation, and video processing in general, has been historically dominated by methods that rely on the temporal consistency and redundancy in consecutive video frames. When temporal smoothness is suddenly broken, such as when an object is occluded, the result of these methods can deteriorate significantly. This paper explores the orthogonal approach of processing each frame independently, i.e. disregarding temporal information. In particular, it tackles the task of semi-supervised video object segmentation: the separation of an object from the background in a video, given its mask in the first frame. We present Semantic One-Shot Video Object Segmentation (OSVOS ), based on a fully-convolutional neural network architecture that is able to successively transfer generic semantic information, learned on ImageNet, to the task of foreground segmentation, and finally to learning the appearance of a single annotated object of the test sequence (hence one shot). We show that instance-level semantic information, when combined effectively, can dramatically improve the results of our previous method, OSVOS. We perform experiments on two recent single-object video segmentation databases, which show that OSVOS is both the fastest and most accurate method in the state of the art. Experiments on multi-object video segmentation show that OSVOS obtains competitive results.

LIP: Learning Instance Propagation for Video Object Segmentation

“…RGMP [54] cannot be successfully trained with static image dataset and DAVIS dataset alone for mask propagation. It has created a large number of synthetic video training data from Pascal VOC [11,12], ECSSD [49] and MSRA10K [7] DAVIS 2017 benchmark, we exclude PReMVOS [38] and OSVOS+ [39] as they both use multiple specialized networks in multiple processes to refine their results. For DAVIS 2016, we compare with OnAVOS [52], FAVOS [5], OSVOS [3], MSK [42], PML [4], SFL [6], OSMN [57], CTN [27] and VPN [26].…”

RANet: Ranking Attention Network for Fast Video Object Segmentation

“…However, OL based methods are computationally expensive for practical applications. In this work, we solve the VOS problem with a very fast network that obtains a competitive accuracy at a speed of 30 FPS on DAVIS 16 , 130 ∼ 400 times faster than previous OL based methods [3,37,40,50].…”

“…To evaluate our RANet trained with static images, we compare it with some methods [22,23,36,40,47] without using DAVIS training set. For multi-object VOS, we compare with some state-of-theart offline methods [3,9,19,50,59], and also list results of some OL based methods [1,3,19,37,50] for reference. Results on DAVIS 16 -val.…”

“…But please note that SiamMask performs badly on objective metrics, e.g., 70.0% at J &F, 15.5 points lower than our RANet. Even when compared with the state-of-the-art OL based methods such as OSVOS-S [37] Table 1, we list the average time of different methods processing a frame of 480p resolution. Note that the proposed RANet spends 33 milliseconds on each frame, much faster than most of the previous methods.…”

“…Early VOS methods [3,37,40,50] mainly resort to online learning (OL) techniques which fine-tune a pre-trained classifier on its first frame. Matching or propagation based methods have also been proposed for VOS.…”