Measuring the Importance of Temporal Features in Video Saliency

Tangemann, Matthias; Kümmerer, Matthias; Wallis, Thomas S. A.; Bethge, Matthias

doi:10.1007/978-3-030-58604-1_40

Cited by 7 publications

(2 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Their conclusion was that better predictions were obtained by considering the eyes and gaze of people in the image. More recent work in video saliency uses deep learning to better mimic human perception [GC18] but predicting the spectator's gaze while viewing cinematographic contents remains a challenging task [TKWB20], further complicated by high level narrative engagement [LLMS15].…”

Section: Eye-tracementioning

confidence: 99%

Film Directing for Computer Games and Animation

Ronfard

2021

Computer Graphics Forum

View full text Add to dashboard Cite

Over the last forty years, researchers in computer graphics have proposed a large variety of theoretical models and computer implementations of a virtual film director, capable of creating movies from minimal input such as a screenplay or storyboard. The underlying film directing techniques are also in high demand to assist and automate the generation of movies in computer games and animation. The goal of this survey is to characterize the spectrum of applications that require film directing, to present a historical and up-to-date summary of research in algorithmic film directing, and to identify promising avenues and hot topics for future research.

show abstract

Section: Eye-tracementioning

confidence: 99%

Film Directing for Computer Games and Animation

Ronfard

2021

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…Specific methods for the dynamic case have been studied [ 28 , 29 , 30 , 31 , 32 , 33 ] and, very recently, unified image-video approaches [ 34 ] proposed, but only in the context of spatial salience. For gaze prediction, temporal features are found to be of key importance in rare events, so spatial static features can explain gaze in most cases [ 35 ]. At the same time, features derived from deep learning models exploiting temporal information have been found to benefit gaze estimation over using static-only features [ 36 ].…”

Section: Related Workmentioning

confidence: 99%

Glimpse: A Gaze-Based Measure of Temporal Salience

Traver

Zorío

Leiva

2021

Sensors

View full text Add to dashboard Cite

Temporal salience considers how visual attention varies over time. Although visual salience has been widely studied from a spatial perspective, its temporal dimension has been mostly ignored, despite arguably being of utmost importance to understand the temporal evolution of attention on dynamic contents. To address this gap, we proposed Glimpse, a novel measure to compute temporal salience based on the observer-spatio-temporal consistency of raw gaze data. The measure is conceptually simple, training free, and provides a semantically meaningful quantification of visual attention over time. As an extension, we explored scoring algorithms to estimate temporal salience from spatial salience maps predicted with existing computational models. However, these approaches generally fall short when compared with our proposed gaze-based measure. Glimpse could serve as the basis for several downstream tasks such as segmentation or summarization of videos. Glimpse’s software and data are publicly available.

show abstract