Visual Saliency Modeling with Deep Learning: A Comprehensive Review

Abstract

Visual saliency models mimic the human visual system to gaze towards fixed pixel positions and capture the most conspicuous regions in the scene. They have proved their efficacy in several computer vision applications. This paper provides a comprehensive review of the recent advances in eye fixation prediction and salient object detection, harnessing deep learning. It also provides an overview on multi-modal saliency prediction that considers audio in dynamic scenes. The underlying network structure and loss function for each model are explored to realise how saliency models work. The survey also investigates the inclusion of specific low-level priors in deep learning-based saliency models. The public datasets and evaluation metrics are succinctly introduced. The paper also makes a discussion on the key issues in saliency modeling along with some open problems and growing research directions in the field.

OnMapGaze and GraphGazeD: A Gaze Dataset and a Graph-Based Metric for Modeling Visual Perception Differences in Cartographic Backgrounds Used in Online Map Services

“…Statistical grayscale heatmaps are one of the main products during the generation of an eye-tracking dataset. Such datasets could serve as the main ground truth for the production of saliency models (see e.g., [47][48][49][50][51]). Additionally, it is important to mention that grayscale statistical heatmaps could also be computed for the analysis of mouse tracking behavioral experiments since the nature of the corresponding data is similar (see e.g., [52]).…”