RVOS: End-To-End Recurrent Network for Video Object Segmentation

Ventura, Carles; Bellver, Miriam; Girbau, Andreu; Salvador, Amaia; Marqués, Ferran; Giró-i-Nieto, Xavier

doi:10.1109/cvpr.2019.00542

Cited by 221 publications

(127 citation statements)

References 33 publications

Supporting

Mentioning

117

Contrasting

Unclassified

Order By: Relevance

“…Test-dev set of DAVIS 17 . In Table 3 we report the performance comparison with the recent instance-level ZVOS method, RVOS [63], on the DAVIS 17 test-dev set. We can find that AGNN significantly outperforms RVOS over most evaluation criteria.…”

Section: Quantitative Performancementioning

confidence: 99%

Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks

Yang

Shen

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

266

133

View full text Add to dashboard Cite

This work proposes a novel attentive graph neural network (AGNN) for zero-shot video object segmentation (ZVOS). The suggested AGNN recasts this task as a process of iterative information fusion over video graphs. Specifically, AGNN builds a fully connected graph to efficiently represent frames as nodes, and relations between arbitrary frame pairs as edges. The underlying pair-wise relations are described by a differentiable attention mechanism. Through parametric message passing, AGNN is able to efficiently capture and mine much richer and higher-order relations between video frames, thus enabling a more complete understanding of video content and more accurate foreground estimation. Experimental results on three video segmentation datasets show that AGNN sets a new state-ofthe-art in each case. To further demonstrate the generalizability of our framework, we extend AGNN to an additional task: image object co-segmentation (IOCS). We perform experiments on two famous IOCS datasets and observe again the superiority of our AGNN model. The extensive experiments verify that AGNN is able to learn the underlying semantic/appearance relationships among video frames or related images, and discover the common objects.

show abstract

Section: Quantitative Performancementioning

confidence: 99%

Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks

Yang

Shen

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

266

133

View full text Add to dashboard Cite

show abstract

“…Differentiable Mask Matching For the feature extractor f θ , we use a COCO-pretrained Mask R-CNN with a ResNet-50-FPN backbone. We also try a ResNet-101 backbone of which the weights are initialized from the released model of RVOS [37]. We denote this model as DMM-Net+.…”

Section: Mask Proposal Generationmentioning

confidence: 99%

DMM-Net: Differentiable Mask-Matching Network for Video Object Segmentation

Zeng

Liao

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

In this paper, we propose the differentiable maskmatching network (DMM-Net) for solving the video object segmentation problem where the initial object masks are provided. Relying on the Mask R-CNN backbone, we extract mask proposals per frame and formulate the matching between object templates and proposals at one time step as a linear assignment problem where the cost matrix is predicted by a CNN. We propose a differentiable matching layer by unrolling a projected gradient descent algorithm in which the projection exploits the Dykstra's algorithm. We prove that under mild conditions, the matching is guaranteed to converge to the optimum. In practice, it performs similarly to the Hungarian algorithm during inference. Meanwhile, we can back-propagate through it to learn the cost matrix. After matching, a refinement head is leveraged to improve the quality of the matched mask. Our DMM-Net achieves competitive results on the largest video object segmentation dataset YouTube-VOS. On DAVIS 2017, DMM-Net achieves the best performance without online learning on the first frames. Without any fine-tuning, DMM-Net performs comparably to state-of-the-art methods on SegTrack v2 dataset. At last, our matching layer is very simple to implement; we attach the PyTorch code (< 50 lines) in the supplementary material. Our code is released at https://github.com/ZENGXH/DMM_Net.

show abstract

“…As PointNet is an effective architecture to extract features from unordered points [37], a PointNet layer is introduced to SFN for feature extraction. In addition, impelled by the success of RNN in video recognition [43][44][45][46], we utilize RNNs to fuse features among different scan lines to acquire 3D contexts. Specifically, RNNs in our SFN are implemented as two single-layer Long Short-Term Memory (LSTM) networks [47].…”

Section: Spatial Fusion Networkmentioning

confidence: 99%

A Deep Learning-Based Method for Overhead Contact System Component Recognition Using Mobile 2D LiDAR

Chen

Lin

et al. 2020

Sensors

View full text Add to dashboard Cite

The overhead contact system (OCS) is a critical railway infrastructure for train power supply. Periodic inspections, aiming at acquiring the operational condition of the OCS and detecting problems, are necessary to guarantee the safety of railway operations. One of the OCS inspection means is to analyze data of point clouds collected by mobile 2D LiDAR. Recognizing OCS components from the collected point clouds is a critical task of the data analysis. However, the complex composition of OCS makes the task difficult. To solve the problem of recognizing multiple OCS components, we propose a new deep learning-based method to conduct semantic segmentation on the point cloud collected by mobile 2D LiDAR. Both online data processing and batch data processing are supported because our method is designed to classify points into meaningful categories of objects scan line by scan line. Local features are important for the success of point cloud semantic segmentation. Thus, we design an iterative point partitioning algorithm and a module named as Spatial Fusion Network, which are two critical components of our method for multi-scale local feature extraction. We evaluate our method on point clouds where sixteen categories of common OCS components have been manually labeled. Experimental results show that our method is effective in multiple object recognition since mean Intersection-over-Unions (mIoUs) of online data processing and batch data processing are, respectively, 96.12% and 97.17%.well record geometric details of surrounding objects [5]. Therefore, it is appropriate to apply MLS systems to OCS inspections. The operational condition of OCS can be acquired by analyzing the MLS point cloud instead of manual measurement. Recognizing the point cloud of OCS as a critical task of the data analysis has been studied in previous studies (e.g., [6][7][8]). Mobile 2D LiDAR is a special kind of MLS system applied to railway inspections. Figure 1 shows an instance of the mobile 2D LiDAR used to scan the OCS infrastructure. As for this kind of MLS system, a point cloud is built up by integrating points at each 2D scan line. Fully understanding the point cloud is significant for automatic inspections and intelligent diagnoses. Thus, this study focuses on multiple OCS component recognition with mobile 2D LiDAR. However, it is a difficult task because the similarities and the various associations among OCS components make scenes complex [8]. In this case, model-driven and data-driven methods become incompetent because rules and features for recognizing specific objects are difficult to be designed by human beings. Fortunately, the success of deep learning-based image recognition (e.g., [9][10][11]) has promoted the fast development of deep learning-based point cloud segmentation, which provides novel means to understand point cloud with semantics.

show abstract

RVOS: End-To-End Recurrent Network for Video Object Segmentation

Cited by 221 publications

References 33 publications

Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks

Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks

DMM-Net: Differentiable Mask-Matching Network for Video Object Segmentation

A Deep Learning-Based Method for Overhead Contact System Component Recognition Using Mobile 2D LiDAR

Contact Info

Product

Resources

About