A Dilated Inception Network for Visual Saliency Prediction

Abstract

Recently, with the advent of deep convolutional neural networks (DCNN), the improvements in visual saliency prediction research are impressive. One possible direction to approach the next improvement is to fully characterize the multiscale saliency-influential factors with a computationally-friendly module in DCNN architectures. In this work, we proposed an end-to-end dilated inception network (DINet) for visual saliency prediction. It captures multi-scale contextual features effectively with very limited extra parameters. Instead of utilizing parallel standard convolutions with different kernel sizes as the existing inception module, our proposed dilated inception module (DIM) uses parallel dilated convolutions with different dilation rates which can significantly reduce the computation load while enriching the diversity of receptive fields in feature maps. Moreover, the performance of our saliency model is further improved by using a set of linear normalization-based probability distribution distance metrics as loss functions. As such, we can formulate saliency prediction as a probability distribution prediction task for global saliency inference instead of a typical pixel-wise regression problem. Experimental results on several challenging saliency benchmark datasets demonstrate that our DINet with proposed loss functions can achieve state-of-the-art performance with shorter inference time.

Object-level Attention for Aesthetic Rating Distribution Prediction

“…We firstly clarify two similar concepts, attention model and attentionbased deep learning model. Attention models are a class of models that aim to predict task-free saliency, including human fixation prediction [8,12,17,33,40] and salient object detection [13,34,36,45,47]. While attention-based deep learning models are deep learning models that aim to enhance their representational power by predicting weights of intermediate features in a task-specific context.…”

“…The attention weights are projected to the corresponding bounded regions in ascending order so that low-attention regions will be covered by high-attention regions when there is any overlapping. The fixation maps are generated with Yang et al's work [40]. Figure 6: IAA-driven object-level attention maps (labels are omitted) that do not fully align with human attention.…”

“…: Left: proposed object-level attention with labels (object category and attention weights). Right: objectagnostic attention [39,40,48]. Both use higher brightness to denote more important regions.…”

Learning synthetic aperture radar image despeckling without clean data

“…Liu et al 40 planned a multibranch residual module with dilated convolutions to extract multiscale features so that the classification and identification of spacecraft electronic load signals can be solved. Yang et al 41 designed an end-to-end dilated inception network (DINet) to predict visual saliency maps. The dilated inception module of the DINet used dilated convolutions with different dilation rates in parallel, which not only can significantly reduce the computational load but also can enrich the diversity of the receptive field in the features.…”

Cross-Modal Feature Integration Network for Human Eye-Fixation Prediction in RGB-D Images

“…Srinivas et al [36] devised a fully CNN for predicting human eye fixation. Yang et al [37] proposed a dilated inception network for visual saliency prediction. Lv et al [38] developed an attention-based fusion network for human eye-fixation prediction in 3-D images.…”