CNNs-Based RGB-D Saliency Detection via Cross-View Transfer and Multiview Fusion

Abstract

Salient object detection from RGB-D images aims to utilize both the depth view and RGB view to automatically localize objects of human interest in the scene. Although a few earlier efforts have been devoted to the study of this paper in recent years, two major challenges still remain: 1) how to leverage the depth view effectively to model the depth-induced saliency and 2) how to implement an optimal combination of the RGB view and depth view, which can make full use of complementary information among them. To address these two challenges, this paper proposes a novel framework based on convolutional neural networks (CNNs), which transfers the structure of the RGB-based deep neural network to be applicable for depth view and fuses the deep representations of both views automatically to obtain the final saliency map. In the proposed framework, the first challenge is modeled as a cross-view transfer problem and addressed by using the task-relevant initialization and adding deep supervision in hidden layer. The second challenge is addressed by a multiview CNN fusion model through a combination layer connecting the representation layers of RGB view and depth view. Comprehensive experiments on four benchmark datasets demonstrate the significant and consistent improvements of the proposed approach over other state-of-the-art methods.

Adaptive Fusion for RGB-D Salient Object Detection

“…Table II and Fig. 4 show that all deep learning based approaches outperform traditional methods by a great margin; and endto-end frameworks, including PCA [19] and our approach, are superior to multi-stage methods such as CTMF [17] and MPCI [18]. Moreover, benefited from our fusion scheme and edge-preserving loss, the proposed method consistently improves the F-measure and MAE achieved by PCA on all three datasets, especially on NLPR where accurate depth data are collected by Kinect.…”

“…For a fair comparison to state-of-the-arts, we utilize the same data split as in [17]. The training set contains 1400 samples from the NJUD dataset and 650 samples from NLPR.…”

“…As done in previous works [17]- [19], β 2 is set to be 0.3 for emphasizing the importance of precision. We compare two kinds of F-measure scores, which are the maximum Fmeasure and the mean F-measure, respectively.…”

Boosting Occluded Image Classification via Subspace Decomposition-Based Estimation of Deep Features

“…For the experiments on the Caltech-101 dataset, the procedures of l 2 normalization (the step (5) in the training phase and the steps (3) and (6) in the testing phase in Algorithm 1) are adopted. For the experiments on the ILSVRC2012 dataset, these procedures are skipped, since the original softmax classifier of the base CNN is not trained over the l 2 normalized DFVs.…”

Video Synchronization Based on Projective-Invariant Descriptor

“…The critical goal for video synchronization is to establish temporal correspondences among frames of two input videos, i.e., a reference video and a video to be synchronized. The applications of video synchronization cover a wide range of video anal-ysis tasks [2][3][4][5][6][7][8], such as video surveillance, target identification, human action recognition, saliency detection and fusion.…”