Affective image classification using features inspired by psychology and art theory

Abstract

Images can affect people on an emotional level. Since the emotions that arise in the viewer of an image are highly subjective, they are rarely indexed. However there are situations when it would be helpful if images could be retrieved based on their emotional content. We investigate and develop methods to extract and combine low-level features that represent the emotional content of an image, and use these for image emotion classification. Specifically, we exploit theoretical and empirical concepts from psychology and art theory to extract image features that are specific to the domain of artworks with emotional expression. For testing and training, we use three data sets: the International Affective Picture System (IAPS); a set of artistic photography from a photo sharing site (to investigate whether the conscious use of colors and textures displayed by the artists improves the classification); and a set of peer rated abstract paintings to investigate the influence of the features and ratings on pictures without contextual content. Improved classification results are obtained on the International Affective Picture System (IAPS), compared to state of the art work.

Analyzing Emotional Semantics of Abstract Art Using Low-Level Image Features

“…This conforms to the previous research using features based on color theories for image emotions' classification [9,16], and to the study where edge and texture features were favored for different art styles' recognition [14].…”

“…[4,9,16,18,19]). Most of the works developed features that are specific to the domains related to art and color theories, which lacks generality and makes it difficult for researchers who are unfamiliar with computer vision algorithms to perform image analysis on their own data [15].…”

“…Similar to [9], we choose abstract art images as our dataset since they usually contain little contextual information that can distort the observer's emotions. However, the features used in [9] are designed on art and color theories and thus are deficient in generality.…”

“…Similar to [9], we choose abstract art images as our dataset since they usually contain little contextual information that can distort the observer's emotions. However, the features used in [9] are designed on art and color theories and thus are deficient in generality. In contrast, we seek to bridge the emotional gap by utilizing a large variety of conventional low-level image features extracted from raw pixels and compound image transforms, which have been widely used in computer vision and machine learning problems.…”

Interpretable aesthetic features for affective image classification

“…Researchers proposed lots of low-level visual features such as SIFT and wavelet textures, and used traditional learning tools to bridge the gap between low-level visual features and high-level semantics, e.g., emotions [1,2]. J. Machajdik summarized more than 100 visual features to perform affective image classification [1]. However, the lowlevel nature of features means they are generally not interpretable, and people cannot know why such a set of features induce a particular emotion.…”