Co-Saliency Detection for RGBD Images Based on Multi-Constraint Feature Matching and Cross Label Propagation

Abstract

Abstract-Co-saliency detection aims at extracting the common salient regions from an image group containing two or more relevant images. It is a newly emerging topic in computer vision community. Different from the most existing co-saliency methods focusing on RGB images, this paper proposes a novel co-saliency detection model for RGBD images, which utilizes the depth information to enhance identification of co-saliency. First, the intra saliency map for each image is generated by the single image saliency model, while the inter saliency map is calculated based on the multi-constraint feature matching, which represents the constraint relationship among multiple images. Then, the optimization scheme, namely Cross Label Propagation (CLP), is used to refine the intra and inter saliency maps in a cross way. Finally, all the original and optimized saliency maps are integrated to generate the final co-saliency result. The proposed method introduces the depth information and multiconstraint feature matching to improve the performance of cosaliency detection. Moreover, the proposed method can effectively exploit any existing single image saliency model to work well in co-saliency scenarios. Experiments on two RGBD co-saliency datasets demonstrate the effectiveness of our proposed model.

HSCS: Hierarchical Sparsity Based Co-saliency Detection for RGBD Images

“…As a supplement, the multiple images relationship is formulated as pairwise correspondences by using the pairwise reconstruction model with a set of pairwise dictionaries, and the F-measure reaches 0.7628. Combining these two aspects, the hierarchical inter saliency structure can explore a more comprehensive inter-image relationship, and reaches 0.8198 in terms of F-measure, which is superior to most of the existing (co-)saliency detection methods (e.g., DSR [9], SMD [15], DF [40], SCS [47], LRMF [53], and ICS [67]). Finally, the cosaliency detection with energy function refinement achieves the best performance, and the percentage gain reaches 3.7% compared with the inter saliency models.…”

“…The average running time is listed in Table IV. In general, compared with the image saliency detection method, co-saliency detection algorithm often requires more computation time, especially for the matching based methods (such as MCLP [68], ICS [67]). For the three RGBD co-saliency detection methods, under the same conditions, the MCLP method takes 41.03 seconds for one image, the ICS method takes 42.67 seconds, and the proposed HSCS method takes an average of 8.29 seconds to process one image.…”

Review of Visual Saliency Detection With Comprehensive Information

“…Combining the depth cue with inter-image correspondence, RGBD co-saliency detection can be achieved. For this task, there are two commonly used datasets, i.e., RGBD Coseg183 dataset [89] and RGBD Cosal150 dataset [91]. Limited by the data sources, only a few of methods are proposed to achieve RGBD co-saliency detection.…”

“…One is the RGBD Coseg183 dataset [89], which contains 183 RGBD images in total that distributed in 16 image groups. The other one is the RGBD Cosal150 dataset [91], which collects 21 image groups containing a total of 150 RGBD images.…”

Multi-Saliency Aggregation-Based Approach for Insulator Flashover Fault Detection Using Aerial Images

“…In this work, a saliency based on the structure feature which can be obtained by modeling the gradient direction, as well as gradient values of the pixels in the image [17,18], is adopted. Similar to the color contrast method, the structural feature is quantized into eight levels according to the pixel histogram of the image gradient distribution, and the gradient histogram ) ( p h g of each pixel P can be obtained through a local observation window p W .…”