Correspondence Driven Saliency Transfer

Yang, Yi; Shen, Jianbing; Shao, Ling; Porikli, Fatih

doi:10.1109/tip.2016.2601784

Cited by 138 publications

(31 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, they do not 1) consider attentive mechanisms; 2) utilize existing large-scale static fixation datasets; and 3) exhaustively assess their performance over large amount of data. There are some salient object detection models [40,1,11,61,58,60,4,62,21] that attempt to uniformly highlight salient object regions in images or videos. Those models are often task-driven and focus on inferring the main object, instead of investigating the behavior of the HVS during scene free viewing.…”

Section: Computational Models For Fixation Predictionmentioning

confidence: 99%

Revisiting Video Saliency: A Large-Scale Benchmark and a New Model

Yang

Shen

Guo

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

Self Cite

222

View full text Add to dashboard Cite

In this work, we contribute to video saliency research in two ways. First, we introduce a new benchmark for predicting human eye movements during dynamic scene freeviewing, which is long-time urged in this field. Our dataset, named DHF1K (Dynamic Human Fixation), consists of 1K high-quality, elaborately selected video sequences spanning a large range of scenes, motions, object types and background complexity. Existing video saliency datasets lack variety and generality of common dynamic scenes and fall short in covering challenging situations in unconstrained environments. In contrast, DHF1K makes a significant leap in terms of scalability, diversity and difficulty, and is expected to boost video saliency modeling. Second, we propose a novel video saliency model that augments the CNN-LSTM network architecture with an attention mechanism to enable fast, end-to-end saliency learning. The attention mechanism explicitly encodes static saliency information, thus allowing LSTM to focus on learning more flexible temporal saliency representation across successive frames. Such a design fully leverages existing large-scale static fixation datasets, avoids overfitting, and significantly improves training efficiency and testing performance. We thoroughly examine the performance of our model, with respect to state-of-the-art saliency models, on three largescale datasets (i.e., DHF1K, Hollywood2, UCF sports). Experimental results over more than 1.2K testing videos containing 400K frames demonstrate that our model outperforms other competitors.

show abstract

Section: Computational Models For Fixation Predictionmentioning

confidence: 99%

Revisiting Video Saliency: A Large-Scale Benchmark and a New Model

Yang

Shen

Guo

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

Self Cite

222

View full text Add to dashboard Cite

show abstract

“…Meanwhile, some other mechanisms [8,54] have been proposed to adopt some prior knowledge, such as a priori background, to detect salient objects in still images. Most existing models directly calculate salient objects (foreground), and thus, these models can also be called a priori foreground models.…”

Section: A Priori Foreground and A Priori Backgroundmentioning

confidence: 99%

Performance Comparison of Saliency Detection

Zhang

et al. 2018

Advances in Multimedia

View full text Add to dashboard Cite

Saliency detection has attracted significant attention in the field of computer vision technology over years. At present, more than 100 saliency detection models have been proposed. In this paper, a relatively more detailed classification is proposed. Furthermore, we selected 25 models and evaluated their performance using four public image datasets. We also discussed common problems, such as the influence to model performance by prior information and multiple objects. Finally, we offered future research directions.

show abstract

“…The former methods [14], [23], [24], [25] try to predict scene locations where a human observer may fixate. Salient object detection [26], [27], [28] aims at uniformly highlighting the salient regions, which has been shown benefit to a wide range of computer vision applications. More detailed reviews of the saliency models can be found in [29], [30].…”

Section: A Saliency Detectionmentioning

confidence: 99%

Video Salient Object Detection via Fully Convolutional Networks

Yang

Shen

Shao

2018

IEEE Trans. on Image Process.

Self Cite

592

224

View full text Add to dashboard Cite

Abstract-This paper proposes a deep learning model to efficiently detect salient regions in videos. It addresses two important issues: 1) deep video saliency model training with the absence of sufficiently large and pixel-wise annotated video data and 2) fast video saliency training and detection. The proposed deep video saliency network consists of two modules, for capturing the spatial and temporal saliency information, respectively. The dynamic saliency model, explicitly incorporating saliency estimates from the static saliency model, directly produces spatiotemporal saliency inference without time-consuming optical flow computation. We further propose a novel data augmentation technique that simulates video training data from existing annotated image data sets, which enables our network to learn diverse saliency information and prevents overfitting with the limited number of training videos. Leveraging our synthetic video data (150K video sequences) and real videos, our deep video saliency model successfully learns both spatial and temporal saliency cues, thus producing accurate spatiotemporal saliency estimate. We advance the state-of-the-art on the densely annotated video segmentation data set (MAE of .06) and the Freiburg-Berkeley Motion Segmentation data set (MAE of .07), and do so with much improved speed (2 fps with all steps).Index Terms-Video saliency, deep learning, synthetic video data, salient object detection, fully convolutional network.

show abstract

Correspondence Driven Saliency Transfer

Cited by 138 publications

References 44 publications

Revisiting Video Saliency: A Large-Scale Benchmark and a New Model

Revisiting Video Saliency: A Large-Scale Benchmark and a New Model

Performance Comparison of Saliency Detection

Video Salient Object Detection via Fully Convolutional Networks

Contact Info

Product

Resources

About