Salient Object Detection in the Deep Learning Era: An In-Depth Survey

Yang, Yi; Lai, Qiuxia; Fu, Huazhu; Shen, Jianbing; Ling, Haibin; Yang, Ruigang

doi:10.48550/arxiv.1904.09146

Cited by 34 publications

(50 citation statements)

References 144 publications

(332 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Under our setup, the use of saliency estimation [8,77] is well motivated. Most importantly, various unsupervised methods exist to generate the saliency masks.…”

Section: Mining Object Mask Proposalsmentioning

confidence: 99%

Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals

Gansbeke¹,

Vandenhende²,

Georgoulis³

et al. 2021

Preprint

View full text Add to dashboard Cite

Being able to learn dense semantic representations of images without supervision is an important problem in computer vision. However, despite its significance, this problem remains rather unexplored, with a few exceptions that considered unsupervised semantic segmentation on small-scale datasets with a narrow visual domain. In this paper, we make a first attempt to tackle the problem on datasets that have been traditionally utilized for the supervised case. To achieve this, we introduce a novel two-step framework that adopts a predetermined prior in a contrastive optimization objective to learn pixel embeddings. This marks a large deviation from existing works that relied on proxy tasks or end-to-end clustering. Additionally, we argue about the importance of having a prior that contains information about objects, or their parts, and discuss several possibilities to obtain such a prior in an unsupervised manner.Extensive experimental evaluation shows that the proposed method comes with key advantages over existing works. First, the learned pixel embeddings can be directly clustered in semantic groups using K-Means. Second, the method can serve as an effective unsupervised pre-training for the semantic segmentation task. In particular, when finetuning the learned representations using just 1% of labeled examples on PASCAL, we outperform supervised ImageNet pre-training by 7.1% mIoU. The code is available 1 .

show abstract

“…Under our setup, the use of saliency estimation [8,77] is well motivated. Most importantly, various unsupervised methods exist to generate the saliency masks.…”

Section: Mining Object Mask Proposalsmentioning

confidence: 99%

Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals

Gansbeke¹,

Vandenhende²,

Georgoulis³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Salient Object Detection (SOD) has a long history date back to Itti et al's work [22]. The majority of SOD [4], [65] methods is designed to detect pixels that belong to the salient objects without knowing the individual instances. So it is commonly treated as a pixel-wise binary classification problem.…”

Section: B Salient Object Detectionmentioning

confidence: 99%

“…Secondly, in most cases, one has to select the primary subject out of numerous foreground distractors within each frame. To overcome this challenge, Salient Object Detection (SOD) [4], [65] and Fixation Prediction (FP) [67] have been practiced. SOD performs pixel-level foregroundbackground binary classification to discover all objects but fails to discriminate the primary subject from other candidates.…”

Section: Introductionmentioning

confidence: 99%

Horizontal-to-Vertical Video Conversion

Zhu

Zhang

et al. 2021

Preprint

View full text Add to dashboard Cite

“…Furthermore, other models combine various complementary information to enhance the model performance, such as the attention mechanism [3], [5], [33], contour detection [5], [6], [34], and gaze prediction [35]. For a more comprehensive survey please refer to [36]. Although some SOD models can also predict different saliency values for different pixels, they are originally designed for binary SOD and focus more on accurately segmenting out salient objects, hence performing unsatisfactorily for saliency ranking detection.…”

Section: Salient Object Detectionmentioning

confidence: 99%

Instance-Level Relative Saliency Ranking with Graph Reasoning

Liu¹,

Li²,

Zhao³

et al. 2021

Preprint

View full text Add to dashboard Cite

Conventional salient object detection models cannot differentiate the importance of different salient objects. Recently, two works have been proposed to detect saliency ranking by assigning different degrees of saliency to different objects. However, one of these models cannot differentiate object instances and the other focuses more on sequential attention shift order inference. In this paper, we investigate a practical problem setting that requires simultaneously segment salient instances and infer their relative saliency rank order. We present a novel unified model as the first end-to-end solution, where an improved Mask R-CNN is first used to segment salient instances and a saliency ranking branch is then added to infer the relative saliency. For relative saliency ranking, we build a new graph reasoning module by combining four graphs to incorporate the instance interaction relation, local contrast, global contrast, and a high-level semantic prior, respectively. A novel loss function is also proposed to effectively train the saliency ranking branch. Besides, a new dataset and an evaluation metric are proposed for this task, aiming at pushing forward this field of research. Finally, experimental results demonstrate that our proposed model is more effective than previous methods. We also show an example of its practical usage on adaptive image retargeting.

show abstract

Salient Object Detection in the Deep Learning Era: An In-Depth Survey

Cited by 34 publications

References 144 publications

Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals

Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals

Horizontal-to-Vertical Video Conversion

Instance-Level Relative Saliency Ranking with Graph Reasoning

Contact Info

Product

Resources

About