Annotating, Understanding, and Predicting Long-term Video Memorability

Cohendet, Romain; Yadati, Karthik; Duong, Ngoc Q. K.; Demarty, Claire-Hélène

doi:10.1145/3206025.3206056

Cited by 33 publications

(43 citation statements)

References 35 publications

(73 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Could people remember dance segments in the first place, given the relatively p.12 difficult nature of the task with the high visual similarity across segments and the complexity of the actions they were seeing? Average hit rate (HR) was 45.77% (SD=13.25%; and this is consistent with what has been found in previous work exploring video memorability for scenes from movies [HR=46.71%, Cohendet et al, 2018]), and average false alarm rate (FAR) was 12.29% (SD=6.49%). Average d' sensitivity further confirmed basic memory (M=1.11, SD=0.36).…”

Section: Resultssupporting

confidence: 90%

“…These results align with previous findings that we can effectively store dynamic actions in memory (Vicary et al, 2014;Wood, 2007;Urgolites & Wood, 2013). They also align with p.19 previous work on memorability of dynamic videos (Cohendet et al, 2018;Newman et al, 2019).…”

Section: Discussionsupporting

confidence: 92%

“…This sample size was determined before data collection began, averaging across previous image and video memorability experiments (e.g. Bainbridge et al, 2013;Cohendet et al, 2018). This sample size was then fixed to be identical across all experiments reported here.…”

Section: Methodsmentioning

confidence: 99%

See 2 more Smart Citations

What moves us?: The intrinsic memorability of dance

Ongchoco¹,

Chun²,

Bainbridge³

2021

Preprint

View full text Add to dashboard Cite

Actions speak loudly, and our ability to remember and keep track of how other people move is fundamental to our interactions with them. But people move in incredibly complex, varied, and expressive ways, and we can’t remember all this information. Here we explore whether and how we prioritize some movements over others. With static images, previous work has found remarkable consistency in which images are remembered or forgotten across people — and this measure of how likely an image is to be remembered consistently is called its ‘memorability’. We know little, however, about how such a property of ‘memorability’ might influence our more dynamic experiences. Using the test case of a rich and abstract series of actions from dance, we discover memorability as an intrinsic attribute of movement. Across genres, some movements were consistently remembered, regardless of the perceiver. Such memorability was independent of static visual information and was additive (where memorability of constituent moments ultimately contributed to the memorability of the longer sequences). Amongst a comprehensive set of movement and aesthetic attributes, consistency in which movements people remembered was most predicted by subjective memorability, and importantly by both subjective (observer ratings) and objective (optical flow analysis) measures of the scale of motion, such that the less overall motion in a dance segment, the more memorable the movements tended to be. Thus, the dynamic segments that were most memorable were those that effectively created static snapshots for the mind.

show abstract

Section: Resultssupporting

confidence: 90%

Section: Discussionsupporting

confidence: 92%

See 1 more Smart Citation

What moves us?: The intrinsic memorability of dance

Ongchoco¹,

Chun²,

Bainbridge³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Recognition memorability is also robust for other types of items, such as abstract visualisations [4], and specific objects within scenes [5]. This intrinsic property is not limited to static images, with faces shown to be consistently memorable across expression and viewpoint distortions [6], and videos shown to be highly consistent in memory performance for both soundless 10-second movie clips [7], and 3-6 second viral videos [8].…”

Section: A Visual Memorabilitymentioning

confidence: 99%

The Influence of Audio on Video Memorability with an Audio Gestalt Regulated Video Memorability System

Sweeney

Healy

Smeaton

2021

2021 International Conference on Content-Based Multimedia Indexing (CBMI)

View full text Add to dashboard Cite

Memories are the tethering threads that tie us to the world, and memorability is the measure of their tensile strength. The threads of memory are spun from fibres of many modalities, obscuring the contribution of a single fibre to a thread's overall tensile strength. Unfurling these fibres is the key to understanding the nature of their interaction, and how we can ultimately create more meaningful media content. In this paper, we examine the influence of audio on video recognition memorability, finding evidence to suggest that it can facilitate overall video recognition memorability rich in high-level (gestalt) audio features. We introduce a novel multimodal deep learning-based late-fusion system that uses audio gestalt to estimate the influence of a given video's audio on its overall short-term recognition memorability, and selectively leverages audio features to make a prediction accordingly. We benchmark our audio gestalt based system on the Memento10k short-term video memorability dataset, achieving top-2 state-of-the-art results.

show abstract

“…It represents the first attempt to build a large-scale corpus regarding video visual memory, which were typically limited to a few hundred of samples in previous works. Added to this, videos were not collected from TRECVID or Hollywood-like films [21,22], but extracted from raw footage originally designed to serve as generic material for professional editing of films or advertisements. Clips are of high quality (HD) and are provided as short soundless videos of 7 s in .webm format, with a bitrate of 3000 kps for 24 fps.…”

Section: Videomemmentioning

confidence: 99%

Topic-Oriented Text Features Can Match Visual Deep Models of Video Memorability

et al. 2021

View full text Add to dashboard Cite

Not every visual media production is equally retained in memory. Recent studies have shown that the elements of an image, as well as their mutual semantic dependencies, provide a strong clue as to whether a video clip will be recalled on a second viewing or not. We believe that short textual descriptions encapsulate most of these relationships among the elements of a video, and thus they represent a rich yet concise source of information to tackle the problem of media memorability prediction. In this paper, we deepen the study of short captions as a means to convey in natural language the visual semantics of a video. We propose to use vector embeddings from a pretrained SBERT topic detection model with no adaptation as input features to a linear regression model, showing that, from such a representation, simpler algorithms can outperform deep visual models. Our results suggest that text descriptions expressed in natural language might be effective in embodying the visual semantics required to model video memorability.

show abstract

Annotating, Understanding, and Predicting Long-term Video Memorability

Cited by 33 publications

References 35 publications

What moves us?: The intrinsic memorability of dance

What moves us?: The intrinsic memorability of dance

The Influence of Audio on Video Memorability with an Audio Gestalt Regulated Video Memorability System

Topic-Oriented Text Features Can Match Visual Deep Models of Video Memorability

Contact Info

Product

Resources

About