An Empirical Study of Detection-Based Video Instance Segmentation

Wang, Qiang; Yi, He; Yang, Xiaoyun; Zhao, Yang; Torr, Philip H. S.

doi:10.1109/iccvw.2019.00089

Cited by 15 publications

(15 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the release of the YouTube-VIS dataset, video instance segmentation has attracted great attention in the computer vision community, arising a series of algorithms recently [49,4,1,2,27,30,41,11]. MaskTrack R-CNN [49] is the first unified model for video instance segmentation.…”

Section: Related Workmentioning

confidence: 99%

Occluded Video Instance Segmentation: A Benchmark

Qi¹,

Gao²,

Hu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Can our video understanding systems perceive objects when a heavy occlusion exists in a scene?To answer this question, we collect a large scale dataset called OVIS for occluded video instance segmentation, that is, to simultaneously detect, segment, and track instances in occluded scenes. OVIS consists of 296k high-quality instance masks from 25 semantic categories, where object occlusions usually occur. While our human vision systems can understand those occluded instances by contextual reasoning and association, our experiments suggest that current video understanding systems are not satisfying. On the OVIS dataset, the highest AP achieved by state-of-the-art algorithms is only 14.4, which reveals that we are still at a nascent stage for understanding objects, instances, and videos in a real-world scenario. Moreover, to complement missing object cues caused by occlusion, we propose a plugand-play module called temporal feature calibration. Built upon MaskTrack R-CNN and SipMask, we report an AP of 15.2 and 15.0 respectively. The OVIS dataset is released at http://songbai.site/ovis, and the project code will be available soon.

show abstract

Section: Related Workmentioning

confidence: 99%

Occluded Video Instance Segmentation: A Benchmark

Qi¹,

Gao²,

Hu³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…VIS requires classifying, segmenting, and tracking visual instances over all frames in a given video. With the introduction of YouTube-VIS-2019 dataset [72], tremendous progresses [19,60,41,17] have been made in tackling this challenging task. As a representative method, MaskTrack R-CNN [72] extends the two-stage instance segmentation model Mask R-CNN with a pair-wise identity branch to solve the instance association sub-task in VIS.…”

Section: Related Workmentioning

confidence: 99%

Crossover Learning for Fast Online Video Instance Segmentation

Yang¹,

Fang²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Modeling temporal visual context across frames is critical for video instance segmentation (VIS) and other video understanding tasks. In this paper, we propose a fast online VIS model named CrossVIS. For temporal information modeling in VIS, we present a novel crossover learning scheme that uses the instance feature in the current frame to pixel-wisely localize the same instance in other frames. Different from previous schemes, crossover learning does not require any additional network parameters for feature enhancement. By integrating with the instance segmentation loss, crossover learning enables efficient crossframe instance-to-pixel relation learning and brings costfree improvement during inference. Besides, a global balanced instance embedding branch is proposed for more accurate and more stable online instance association. We conduct extensive experiments on three challenging VIS benchmarks, i.e., YouTube-VIS-2019, OVIS, and YouTube-VIS-2021 to evaluate our methods. To our knowledge, CrossVIS achieves state-of-the-art performance among all online VIS methods and shows a decent trade-off between latency and accuracy. Code will be available to facilitate future research.

show abstract

“…The works in this category can be divided into two categories, i) treats instance segmentation and tracking as two separate tasks [26,33] and ii) has end to end trainable pipeline [3,6,1,21,40,22,12,11]. The methods in first category fine-tune each stage individually and are able to achieve good results.…”

Section: Video Instance Segmentationmentioning

confidence: 99%

MSN: Efficient Online Mask Selection Network for Video Instance Segmentation

Goel,

Li,

Garg

et al. 2021

Preprint

View full text Add to dashboard Cite

In this work we present a novel solution for Video Instance Segmentation(VIS), that is automatically generating instance level segmentation masks along with object class and tracking them in a video. Our method improves the masks from segmentation and propagation branches in an online manner using the Mask Selection Network (MSN) hence limiting the noise accumulation during mask tracking. We propose an effective design of MSN by using patch-based convolutional neural network. The network is able to distinguish between very subtle differences between the masks and choose the better masks out of the associated masks accurately. Further we make use of temporal consistency and process the video sequences in both forward and reverse manner as a post processing step to recover lost objects. The proposed method can be used to adapt any video object segmentation method for the task of VIS. Our method achieves a score of 49.1 mAP on 2021 YouTube-VIS Challenge and was ranked third place among more than 30 global teams.

show abstract

An Empirical Study of Detection-Based Video Instance Segmentation

Cited by 15 publications

References 24 publications

Occluded Video Instance Segmentation: A Benchmark

Occluded Video Instance Segmentation: A Benchmark

Crossover Learning for Fast Online Video Instance Segmentation

MSN: Efficient Online Mask Selection Network for Video Instance Segmentation

Contact Info

Product

Resources

About