MemCat: a new category-based image set quantified on memorability

Abstract

Images differ in their memorability in consistent ways across observers. What makes an image memorable is not fully understood to date. Most of the current insight is in terms of high-level semantic aspects, related to the content. However, research still shows consistent differences within semantic categories, suggesting a role for factors at other levels of processing in the visual hierarchy. To aid investigations into this role as well as contributions to the understanding of image memorability more generally, we present MemCat. MemCat is a category-based image set, consisting of 10K images representing five broader, memorability-relevant categories (animal, food, landscape, sports, and vehicle) and further divided into subcategories (e.g., bear). They were sampled from existing source image sets that offer bounding box annotations or more detailed segmentation masks. We collected memorability scores for all 10 K images, each score based on the responses of on average 99 participants in a repeat-detection memory task. Replicating previous research, the collected memorability scores show high levels of consistency across observers. Currently, MemCat is the second largest memorability image set and the largest offering a category-based structure. MemCat can be used to study the factors underlying the variability in image memorability, including the variability within semantic categories. In addition, it offers a new benchmark dataset for the automatic prediction of memorability scores (e.g., with convolutional neural networks). Finally, MemCat allows the study of neural and behavioral correlates of memorability while controlling for semantic category.

Memoir Dataset: Quantifying Image Memorability in Adolescents

“…Isola, Xiao, et al (2011) introduced their large-scale visual memory game used to test and quantify the memorability of images and hosted it on Amazon Mechanical Turk to allow for crowdsourcing. This experiment has since been replicated many times (Goetschalckx & Wagemans, 2019;Isola et al, 2013;Khosla et al, 2015) and has formed the foundations of measuring image memorability. Memorability is a critical property of images as it has been found to be an automatic, unconscious form of memory that is disparate from other cognitive processes that influence memory behaviour (Bainbridge, 2020;Bainbridge et al, 2017;Mohsenzadeh, Mullin, Oliva, & Pantazis, 2019).…”

“…The stimuli in this study were selected from MemCat (Goetschalckx & Wagemans, 2019), a recently published image memorability dataset. The MemCat dataset is publicly available and includes 10,000 images that have been annotated with adult memorability scores.…”

“…A special feature of the MemCat dataset (compared to other image memorability datasets (Bainbridge, Isola, & Oliva, 2013;Borkin et al, 2013;Bylinskii et al, 2015;Isola et al, 2013;Khosla et al, 2015)) is its hierarchical category structure: it features five main categories (animals, food, landscapes, sports, and vehicles), each with at least 20 subcategories (such as sheep, salad, waterfall, basketball, and airplane). This allowed us to robustly analyze the effects of these categories on memorability, as done previously in adults (Goetschalckx & Wagemans, 2019;Isola, Parikh, et al, 2011;Isola et al, 2013;Khosla et al, 2012;Perera et al, 2019). For our adolescent memorability dataset, which we call the Memoir dataset, we randomly sampled 1,000 images from the MemCat dataset equally from each category and from the full range of memorability scores provided in MemCAT.…”

“…To understand memorability, the visual memory game described above has been used to assign memorability scores to thousands of images, and it has been shown that certain image features can be extracted and linked to memorability. Specific object categories (people, animals, and vehicles) as well as scene categories (indoor scenes) are more memorable than others (Goetschalckx & Wagemans, 2019;Isola, Parikh, Torralba, & Oliva, 2011;Isola et al, 2013;Khosla, Xiao, Isola, Torralba, & Oliva, 2012;Perera, Tal, & Zelnik-Manor, 2019). In fact, Isola et al (2013) have reported that the scene category captures most of what makes an image memorable.…”

“…Saliency and contrast have been shown to be important factors, meaning images with specific points of focus are more memorable (Isola, Parikh, et al, 2011;Mancas & Le Meur, 2013). Interestingly, image aesthetics and perceived popularity have shown little to no correlation with memorability, despite being correlated to each other (Goetschalckx & Wagemans, 2019;Isola, Parikh, et al, 2011;Khosla et al, 2012). Similarly, participants' perceived estimates of image memorability have a low correlation with their true memorability (Isola, Xiao, et al, 2011), as most people wrongly assume more beautiful and aesthetic images will be easier to remember.…”