How to Evaluate Foreground Maps

Margolin, Ran; Zelnik-Manor, Lihi; Tal, Ayellet

doi:10.1109/cvpr.2014.39

Cited by 637 publications

(278 citation statements)

References 16 publications

Supporting

Mentioning

259

Contrasting

Unclassified

Order By: Relevance

“…Also, we compare our method with 24 state-of-the-art methods using six performance metrics, including the traditional measures, e.g., precision-recall curve and mean absolute error, and the recently proposed weighted Fmeasure [32]. In the experiments, our SMD-based algorithm achieves competitive results in comparison with other leading methods.…”

Section: (Lrr Ulr and Slr)mentioning

confidence: 99%

Salient Object Detection via Structured Matrix Decomposition

Peng

Ling

et al. 2017

IEEE Trans. Pattern Anal. Mach. Intell.

335

210

View full text Add to dashboard Cite

Abstract-Low-rank recovery models have shown potential for salient object detection, where a matrix is decomposed into a low-rank matrix representing image background and a sparse matrix identifying salient objects. Two deficiencies, however, still exist. First, previous work typically assumes the elements in the sparse matrix are mutually independent, ignoring the spatial and pattern relations of image regions. Second, when the low-rank and sparse matrices are relatively coherent, e.g., when there are similarities between the salient objects and background or when the background is complicated, it is difficult for previous models to disentangle them. To address these problems, we propose a novel structured matrix decomposition model with two structural regularizations: (1) a tree-structured sparsity-inducing regularization that captures the image structure and enforces patches from the same object to have similar saliency values, and (2) a Laplacian regularization that enlarges the gaps between salient objects and the background in feature space. Furthermore, high-level priors are integrated to guide the matrix decomposition and boost the detection. We evaluate our model for salient object detection on five challenging datasets including single object, multiple objects and complex scene images, and show competitive results as compared with 24 state-of-the-art methods in terms of seven performance metrics.

show abstract

Section: (Lrr Ulr and Slr)mentioning

confidence: 99%

Salient Object Detection via Structured Matrix Decomposition

Peng

Ling

et al. 2017

IEEE Trans. Pattern Anal. Mach. Intell.

335

210

View full text Add to dashboard Cite

show abstract

“…and subjective scores were inverted by subtracting the scores from the maximum subject score of 5 to obtain positive correlation scores. For the W F β measure [10], which requires the groundtruth to be a binary mask, we threshold the ground-truth saliency map by its standard deviation as suggested in [32]. For the M AE metric, instead of using a binary ground-truth map as in [11], we use a real-valued ground-truth saliency map since, by definition, the M AE can be computed for two real-valued maps.…”

Section: Subjective Evaluation Of Va Modelsmentioning

confidence: 99%

“…Non-shuffled Metrics (c) ( However, the AUC metrics including the sAU C suffer from another notable flaw known as the interpolation flaw (described in detail in [10]). As seen in Figure 2, AU C Borji , AU C Judd and sAU C are less sensitive to false-positives.…”

Section: Existing Metrics and Their Shortcomingsmentioning

confidence: 99%

“…The AU C Borji [3] and its derivative sAU C [3] (j) SIM [13] (k) sN SS [7] (l) W F β [10] 0.4 0.6 0. Most of the existing metrics along with the widely used AUC based metrics like sAU C [3], AU C Borji [3] and AU C Judd [13] do not show high correlation with subjective scores.…”

Section: Subjective Evaluation Of Va Modelsmentioning

confidence: 99%

“…The MIT saliency benchmark project [9] is an up-to-date online benchmarking resource that lists the performance of all the recent state-of-the-art VA models using seven popular evaluation metrics that are a combination of those used in [3] and [4]. The metrics used by the MIT Saliency Benchmark [9] along with recently proposed metrics W F β [10] and sN SS [7] in addition to a baseline metric M AE [11] are listed in Table I along with the categories they belong to, as well as the type of ground-truth used.…”

Section: Existing Metrics and Their Shortcomingsmentioning

confidence: 99%

See 2 more Smart Citations

A Locally Weighted Fixation Density-Based Metric for Assessing the Quality of Visual Saliency Predictions

Gide¹,

Karam²

2016

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

With the increased focus on visual attention (VA) in the last decade, a large number of computational visual saliency methods have been developed over the past few years. These models are traditionally evaluated by using performance evaluation metrics that quantify the match between predicted saliency and fixation data obtained from eye-tracking experiments on human observers. Though a considerable number of such metrics have been proposed in the literature, there are notable problems in them. In this work, we discuss shortcomings in existing metrics through illustrative examples and propose a new metric that uses local weights based on fixation density which overcomes these flaws. To compare the performance of our proposed metric at assessing the quality of saliency prediction with other existing metrics, we construct a ground-truth subjective database in which saliency maps obtained from 17 different VA models are evaluated by 16 human observers on a 5-point categorical scale in terms of their visual resemblance with corresponding ground-truth fixation density maps obtained from eyetracking data. The metrics are evaluated by correlating metric scores with the human subjective ratings.The correlation results show that the proposed evaluation metric outperforms all other popular existing metrics. Additionally, the constructed database and corresponding subjective ratings provide an insight into which of the existing metrics and future metrics are better at estimating the quality of saliency prediction and can be used as a benchmark.

show abstract

Dual‐branch mutual assistance network for salient object detection

Yao

Wang

2021

Int J Intell Syst

View full text Add to dashboard Cite

To maintain the boundaries of salient objects in the detection results, some methods start to apply additional edge labels to train their networks to learn detailed information. These methods have made gratifying progress. However, since the quality of the saliency maps will be affected by boundary features, it is still worth exploring how to extract effective boundary features and fuse them with semantic features. In this

show abstract

How to Evaluate Foreground Maps

Cited by 637 publications

References 16 publications

Salient Object Detection via Structured Matrix Decomposition

Salient Object Detection via Structured Matrix Decomposition

A Locally Weighted Fixation Density-Based Metric for Assessing the Quality of Visual Saliency Predictions

Dual‐branch mutual assistance network for salient object detection

Contact Info

Product

Resources

About