MoNet: Deep Motion Exploitation for Video Object Segmentation

Abstract

In this paper, we propose a novel MoNet model to deeply exploit motion cues for boosting video object segmentation performance from two aspects, i.e., frame representation learning and segmentation refinement. Concretely, MoNet exploits computed motion cue (i.e., optical flow) to reinforce the representation of the target frame by aligning and integrating representations from its neighbors. The new representation provides valuable temporal contexts for segmentation and improves robustness to various common contaminating factors, e.g., motion blur, appearance variation and deformation of video objects. Moreover, MoNet exploits motion inconsistency and transforms such motion cue into foreground/background prior to eliminate distraction from confusing instances and noisy regions. By introducing a distance transform layer, MoNet can effectively separate motion-inconstant instances/regions and thoroughly refine segmentation results. Integrating the proposed two motion exploitation components with a standard segmentation network, MoNet provides new state-of-the-art performance on three competitive benchmark datasets.

FEELVOS: Fast End-To-End Embedding Learning for Video Object Segmentation

“…Location-sensitive embeddings used to refine an initial foreground prediction are explored in LSE [9]. MoNet [38] exploits optical flow motion cues by feature alignment and a distance transform layer. Using reinforcement learning to estimate a region of interest to be segmented is explored by Han et al [13].…”

See More, Know More: Unsupervised Video Object Segmentation With Co-Attention Siamese Networks

“…For SVOS methods, the target object(s) is provided in the first frame and tracked automatically [60,8,5,68,2,69,64,71] or interactively by users [1] in the subsequent frames. Numerous algorithms were proposed based on graphical models [54], object proposals [46], supertrajectories [61], etc.…”

Global Second-Order Pooling Convolutional Networks

“…Wang et al [33] proposed a global Gaussian distribution embedding network (G 2 DeNet), where one multivariate Gaussian, identified as a symmetric positive definite matrix of covariance matrix and mean vector [20], is plugged at network end. MoNet [38] proposed a sub-matrix square-root layer, making G 2 DeNet to have compact representation. In [3], the first-order information are combined with the second-order one which achieves consistent improvements over the standard bilinear networks on texture recognition.…”