Glimpse: A Gaze-Based Measure of Temporal Salience

Abstract

Temporal salience considers how visual attention varies over time. Although visual salience has been widely studied from a spatial perspective, its temporal dimension has been mostly ignored, despite arguably being of utmost importance to understand the temporal evolution of attention on dynamic contents. To address this gap, we proposed Glimpse, a novel measure to compute temporal salience based on the observer-spatio-temporal consistency of raw gaze data. The measure is conceptually simple, training free, and provides a semantically meaningful quantification of visual attention over time. As an extension, we explored scoring algorithms to estimate temporal salience from spatial salience maps predicted with existing computational models. However, these approaches generally fall short when compared with our proposed gaze-based measure. Glimpse could serve as the basis for several downstream tasks such as segmentation or summarization of videos. Glimpse’s software and data are publicly available.

Saliency-Based Gaze Visualization for Eye Movement Analysis

“…Most gaze data visualization studies discover rationales related to gaze behavior from cognitive control and analyze common eye movements or personal biases of observers [ 24 , 25 ], essential information from AOIs (areas of interest) [ 26 ], the influence of memory and knowledge on eye movements [ 27 ], and social communication [ 28 ]. According to the analysis of visual attention and eye movement, gaze behavior is affected not only by cognitive control but also by visual saliency [ 29 , 30 ]. In order to use visual saliency for gaze behavior analysis, researchers need to statistically calculate saliency feature values in the fixations separately from gaze data visualization [ 31 ] or to utilize tools that show saliency features of visual stimuli [ 15 ].…”