Gravitational models explain shifts on human visual attention

Abstract

Visual attention refers to the human brain’s ability to select relevant sensory information for preferential processing, improving performance in visual and cognitive tasks. It proceeds in two phases. One in which visual feature maps are acquired and processed in parallel. Another where the information from these maps is merged in order to select a single location to be attended for further and more complex computations and reasoning. Its computational description is challenging, especially if the temporal dynamics of the process are taken into account. Numerous methods to estimate saliency have been proposed in the last 3 decades. They achieve almost perfect performance in estimating saliency at the pixel level, but the way they generate shifts in visual attention fully depends on winner-take-all (WTA) circuitry. WTA is implemented by the biological hardware in order to select a location with maximum saliency, towards which to direct overt attention. In this paper we propose a gravitational model to describe the attentional shifts. Every single feature acts as an attractor and the shifts are the result of the joint effects of the attractors. In the current framework, the assumption of a single, centralized saliency map is no longer necessary, though still plausible. Quantitative results on two large image datasets show that this model predicts shifts more accurately than winner-take-all.

HeMoG: A White-Box Model to Unveil the Connection between Saliency Information and Human Head Motion in Virtual Reality

“…Given the head orientation predicted by HeMoG, the FoV can be cropped from the 360 • content. Then, a model for regular 2D videos as the one proposed in [20] can be used on this planar FoV section to predict plausible human gaze scanpaths.…”

“…Finally, for regular 2D videos, [20] recently proposed a gravitational model to generate human-plausible visual scanpaths. We take inspiration from this model to design HeMoG, which, contrary to [20], is built on a 3D-rotational motion description with specific terms related to head/neck fatigue.…”

Deep Learning to See

“…Unlike the other described approaches, the prediction of the focus does not rely on a centralized saliency map, but it acts directly on early representations of basic features organized in spatial maps. Besides the advantage in real-time applications, these models make it possible to characterize patterns of eye movements (such as fixations, saccades and smooth pursuit) and, despite their simplicity, they reach the state of the art in scanpath prediction and proved to predict shifts in visual attention better than the classic winner-take-all [92].…”

Optimal Topology of Vision Transformer for Real-Time Video Action Recognition in an End-To-End Cloud Solution

“…The human brain inspires most deep learning architectures currently employed in cloud-based solutions, e.g., convolutional neural networks [15][16][17], recurrent neural networks, long short-term memory, known as RNN-LSTM [18,19], and attention-based networks [20,21]. For instance, a deep learning convolutional network was implemented for predicting the different stages of Alzheimer's disease, including normal aging, mild cognitive impairment, and Alzheimer's [15][16][17].…”