Motion-Guided Cascaded Refinement Network for Video Object Segmentation

Spatiotemporal CNN for Video Object Segmentation

“…We compare the proposed method with 7 state-of-the-art semi-supervised methods, i.e., CRN [15], OnAVOS [47], OSVOS [3], MSK [38], CTN [23], SegFlow [5], and VPN [22], and 4 state-of-the-art unsupervised methods, namly ARP [28], LVO [45], FSEG [21], and LMP [44] in Table 1.…”

“…We believe that is can be used in our STCNN to further improve the performance. In addition, in contrast to CRN [15] relying on optical flow to render temporal coherence in both training and testing, our method uses a self-supervised strategy to implicitly exploit the temporal coherence without relying on the expensive human annotations of optical flow. The temporal coherence branch is able to capture the dynamic appearance and motion cues of video sequences, pretrained in an adversarial manner from nearly unlimited unlabeled video data.…”

“…We compare the proposed STCNN method to 11 stateof-the-art semi-supervised algorithms, namely BVS [34], JFS [35], SCF [20], MRFCNN [2], LT [25], OSVOS [3], MSK [38], OFL [46], CRN [15], DRL [12], and OnAVOS [47] in Table 2. As shown in Table 2, we observe that the STCNN method produces the best results with 0.796 mean IoU, which surpasses the state-of-the-art results, i.e., MR-FCNN [2] (0.784 mean IoU), with 0.012 mIoU.…”

“…In this paper, we are interested in the semi-supervised VOS task, which can be treated as the label propagation problem through the entire video. To maintain the temporal associations of object segments, optical flow is usually used in most of previous methods [48,46,5,23,44,2,15] to model the pixel consistency across the time for smoothness. However, optical flow annotation requires significant human effort, and estimation is challenging and often inaccurate, and thus it is not always helpful in video segmentation.…”

“…The spatial segmentation branch is a fully convolutional network focusing on segmenting objects based on the learned appearance and motion cues from the temporal coherence branch. Inspired by [15], we design a coarse-to-fine process to sequentially apply a designed attention module on multi-scale feature maps, and concatenate them to produce the final accurate prediction. In this way, the spatial segmentation branch is enforced to gradually concentrate on the object regions, which benefits both training and testing.…”