Weakly Supervised Video Salient Object Detection

Zhao, Wangbo; Zhang, Jing; Li, Long; Barnes, Nick; Liu, Nian; Han, Junwei

doi:10.1109/cvpr46437.2021.01655

Cited by 72 publications

(34 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It greatly reduces the manual labeling cost, but still requires 20% pixel-wise annotations, which would be a large project for large video datasets with much richer scenes. Zhao et al [20] designed a weakly supervised VSOD model via scribble to relieve the burden of pixel-wise labeling. Although scribble annotations can effectively reduce time and capital cost compared with pixel-wise annotations, it still requires expensive, time-consuming labeling.…”

Section: Weakly/semi/un-supervised Video Salient Object Detectionmentioning

confidence: 99%

“…As shown in Table I, we compare our proposed method with existing three image salient object detection models (PoolNet [54], EGNet [57], PAKRN [21]), ten fully supervised VSOD models (SCOM [41], MBN [58], PDB [8], FGRN [42], MGA [11], RCRN [10], SSAV [9], PCSA [43], STVS [44], and DCFN [45]), and four weakly or unsupervised models(SSOD [59], GF [23], SAG [1], WS [20]) . For fair comparison, results of these methods are directly provided by authors or by their trained model and we use the same evaluation codes to test them.…”

Section: H Comparison With the State-of-the-artsmentioning

confidence: 99%

See 1 more Smart Citation

Learning Video Salient Object Detection Progressively from Unlabeled Videos

Xu¹,

Liang²,

Ni³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent deep learning-based video salient object detection (VSOD) has achieved some breakthrough, but these methods rely on expensive annotated videos with pixel-wise annotations, weak annotations, or part of the pixel-wise annotations. In this paper, based on the similarities and the differences between VSOD and image salient object detection (SOD), we propose a novel VSOD method via a progressive framework that locates and segments salient objects in sequence without utilizing any video annotation. To use the knowledge learned in the SOD dataset for VSOD efficiently, we introduce dynamic saliency to compensate for the lack of motion information of SOD during the locating process but retain the same fine segmenting process. Specifically, an algorithm for generating spatiotemporal location labels, which consists of generating high-saliency location labels and tracking salient objects in adjacent frames, is proposed. Based on these location labels, a two-stream locating network that introduces an optical flow branch for video salient object locating is presented. Although our method does not require labeled video at all, the experimental results on five public benchmarks of DAVIS, FBMS, ViSal, VOS, and DAVSOD demonstrate that our proposed method is competitive with fully supervised methods and outperforms the state-of-the-art weakly and unsupervised methods.

show abstract

Section: Weakly/semi/un-supervised Video Salient Object Detectionmentioning

confidence: 99%

Section: H Comparison With the State-of-the-artsmentioning

confidence: 99%

Learning Video Salient Object Detection Progressively from Unlabeled Videos

Xu¹,

Liang²,

Ni³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Guo et al [40] proposed a fast VSOD method by using the principal motion vectors to represent the corresponding motion patterns, and such motion message coupling with the color clues together are fed into the multiclue optimization framework to achieve the spatiotemporal VSOD. Zhao et al [41] proposed a weakly-supervised VSOD model based on eye-fixation annotation. Compared with the fully-supervised VSOD models, the proposed new annotation method dramatically reduces the consumption of time.…”

Section: A Hand-crafted Vsod Approachesmentioning

confidence: 99%

A Novel Long-term Iterative Mining Scheme for Video Salient Object Detection

Chen¹,

Wang²,

Fang³

et al. 2022

Preprint

View full text Add to dashboard Cite

The existing state-of-the-art (SOTA) video salient object detection (VSOD) models have widely followed short-term methodology, which dynamically determines the balance between spatial and temporal saliency fusion by solely considering the current consecutive limited frames. However, the short-term methodology has one critical limitation, which conflicts with the real mechanism of our visual system -a typical longterm methodology. As a result, failure cases keep showing up in the results of the current SOTA models, and the short-term methodology becomes the major technical bottleneck. To solve this problem, this paper proposes a novel VSOD approach, which performs VSOD in a complete long-term way. Our approach converts the sequential VSOD, a sequential task, to a data mining problem, i.e., decomposing the input video sequence to object proposals in advance and then mining salient object proposals as much as possible in an easy-to-hard way. Since all object proposals are simultaneously available, the proposed approach is a complete long-term approach, which can alleviate some difficulties rooted in conventional short-term approaches. In addition, we devised an online updating scheme that can grasp the most representative and trustworthy pattern profile of the salient objects, outputting framewise saliency maps with rich details and smoothing both spatially and temporally. The proposed approach outperforms almost all SOTA models on five widely used benchmark datasets.

show abstract

“…As a sparse and low-cost supervision label, the scribble annotation has received increasing attention for salient object detection and object segmentation in recent works [28,29,30,31,32,16,17]. As is known to all, the scribble annotations are too simple and sparse to convey sufficient information, e.g., object structure and details.…”

Section: Introductionmentioning

confidence: 99%

Scribble-based Boundary-aware Network for Weakly Supervised Salient Object Detection in Remote Sensing Images

Zhou¹,

Xiang²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

Existing CNNs-based salient object detection (SOD) heavily depends on the large-scale pixel-level annotations, which is labor-intensive, time-consuming, and expensive. By contrast, the sparse annotations (e.g., image-level or scribble) become appealing to the salient object detection community. However, few efforts are devoted to learning salient object detection from sparse annotations, especially in the remote sensing field. In addition, the sparse annotation usually contains scanty information, which makes it challenging to train a well-performing model, resulting in its performance largely lagging behind the fully-supervised models. Although some SOD methods adopt some prior cues (e.g., edges) to improve the detection performance, they usually lack targeted discrimination of object boundaries and thus provide saliency maps with poor boundary localization. To this end, in this paper, we propose a novel weaklysupervised salient object detection framework to predict the saliency of remote sensing images from sparse scribble annotations. To implement it, we first construct the scribble-based remote sensing saliency dataset by relabelling an existing large-scale SOD dataset with scribbles, namely S-EOR dataset. After that,

show abstract

Weakly Supervised Video Salient Object Detection

Cited by 72 publications

References 37 publications

Learning Video Salient Object Detection Progressively from Unlabeled Videos

Learning Video Salient Object Detection Progressively from Unlabeled Videos

A Novel Long-term Iterative Mining Scheme for Video Salient Object Detection

Scribble-based Boundary-aware Network for Weakly Supervised Salient Object Detection in Remote Sensing Images

Contact Info

Product

Resources

About