Some Objects Are More Equal Than Others: Measuring and Predicting Importance

Abstract

Abstract. We observe that everyday images contain dozens of objects, and that humans, in describing these images, give different priority to these objects. We argue that a goal of visual recognition is, therefore, not only to detect and classify objects but also to associate with each a level of priority which we call 'importance'. We propose a definition of importance and show how this may be estimated reliably from data harvested from human observers. We conclude by showing that a first-order estimate of importance may be computed from a number of simple image region measurements and does not require access to image meaning.

Learning the Relative Importance of Objects from Tagged Images for Retrieval and Cross-Modal Search

“…Though difficult to state in the absolute, a number of previous studies lend support for both points (von Ahn and Dabbish 2004;Spain and Perona 2008;Einhauser et al 2008;Elazary and Itti 2008;Tatler et al 2005;Hwang and Grauman 2010b), as does our own experimental data.…”

“…On the other hand, other studies demonstrate that the top-down saliency of recognized objects clearly directs a viewer's attention Einhauser et al (2008). Spain and Perona (2008) give a specific definition for importance: an object's importance in an image is the probability that it would be named first by a viewer. The authors devise a model for this concept, and demonstrate that one can predict the importance of named objects in an image via regression on some intuitive image cues (e.g., scale, saliency, etc.).…”

“…Their premise of some objects being more important than others motivates our idea, and, like the ESP game results, their empirical findings about agreement across annotators support the feasibility of learning from human-provided tags. However, the system developed in Spain and Perona (2008) assumes that all examples (including the novel test inputs) are already hand-segmented and labeled by object category, which prevents its use for the retrieval applications considered in this work. In contrast, our algorithm uses unsegmented, unlabeled data for both training and testing.…”

“…In this work we also consider ways to elicit the implied information in textual descriptions of images, and, similar to Spain and Perona (2008) and Hwang and Grauman (2010b), we are interested in the order in which people name objects in a scene. However, our target application is quite distinct.…”

What makes an image memorable?

“…As opposed to other image properties, there are no previous studies that try to quantify individual images in terms of how memorable they are, and there are no computer vision systems that try to predict image memorability. This is contrary to many other photographic properties that have been addressed in the literature such as photo quality [16], saliency [10], attractiveness [15], composition [8,18], color harmony [4], and object importance [22]. Also, there are no databases of photographs calibrated in terms of the degree of memorability for each image.…”