Image memorability is predicted by discriminability and similarity in different stages of a convolutional neural network

Abstract

The features of an image can be represented at multiple levels – from low-level visual properties to high-level meaning. What factors drive some images to be memorable while others are forgettable? Across three behavioral experiments, we addressed this question. In a first behavioral experiment, we combined a convolutional neural network (CNN) with behavioral prospective assignment, by using four CNN layers to select the scene images that each of one hundred participants experience. We found that participants remembered more images when they were assigned to view stimuli selected based on their discriminability in low-level CNN layers, or similarity in high-level layers. A second experiment replicated these results in an independent sample of participants. The third experiment investigated how memorability is influenced when images fall within a single semantic category (houses). We replicated results from the first two experiments at lower levels, but found that similarity predicted memorability at a mid-high level, rather than the highest level observed for scenes from multiple categories. This mid-high level contains representations for objects and object-parts, which are more important for discriminating images from the same category. Together, this research provides evidence that discriminability at different visual levels, modeled using a CNN, predicts image memorability.