Aesthetic Attributes Assessment of Images

Abstract

Figure 1: Aesthetic Attributes Assessment of Images. We predict caption and score of each aesthetic attribute of an image.ABSTRACT Image aesthetic quality assessment has been a relatively hot topic during the last decade. Most recently, comments type assessment (aesthetic captions) has been proposed to describe the general aesthetic impression of an image using text. In this paper, we propose Aesthetic Attributes Assessment of Images, which means the aesthetic attributes captioning. This is a new formula of image aesthetic assessment, which predicts aesthetic attributes captions together with the aesthetic score of each attribute. We introduce a new dataset named DPC-Captions which contains comments of up to 5 aesthetic attributes of one image through knowledge transfer from a full-annotated small-scale dataset. Then, we propose Aesthetic Multi-Attribute Network (AMAN), which is trained on a mixture of fully-annotated small-scale PCCD dataset and weakly-annotated large-scale DPC-Captions dataset. Our AMAN makes full use of transfer learning and attention model in a single framework. The experimental results on our DPC-Captions and PCCD dataset reveal that our method can predict captions of 5 aesthetic attributes together with numerical score assessment of each attribute. We use the evaluation criteria used in image captions to prove that our specially designed AMAN model outperforms traditional CNN-LSTM model and modern SCA-CNN model of image captions.

A Comprehensive Survey on Computational Aesthetic Evaluation of Visual Art Images: Metrics and Challenges

“…Negative bias 8 --Binary label HiddenBeauty [43] 15,000 1-5 4 -5 --Aesthetic score AVA [41] 255,530 1-10 66 Positive bias 210 14 -Distribution AROD [44] 380,000 --Uniform 6868 --Aesthetic score AADB [45] 10,000 1-5 -Normal 5 8 -Binary Attribute PCCD [46] 4,235 1-10 --7 7 29,645 Comments AVA-Reviews [47] 40,000 1-10 66 Positive bias 6 -240,000 Comments AVA-Comments [48] 255,530 1-10 66 Positive bias 6 -1,535,937 Comments DPC-Captions [49] 154 4.5k~13k balanced distribution of professional and consumer photos, with a total of 10, 000 images. Eleven aesthetic attributes and annotators' ID is provided.…”

“…The AVA-Comments [48] and AVA-Reviewers [47] datasets are designed by selecting 255,530 and 40,000 images from AVA dataset and add single sentence comments to describe overall impression. Jin et al [49] built a new dataset named DPC-Captions. It contains 154,384 images and 2,427,483 comments of up to 5 aesthetic attributes using aesthetic knowledge transfer from the fullannotated small-scale PCCD and AVA-Plus datasets, which crawled 330,000 images together with their comments from DPChallenge.com to a large-scale weakly annotated one.…”

“…Beginning with the strong performance of Krizhevsky et al [18] in the image classification, the powerful feature representation learned with a growing amount of datasets, and feasible transfer learning [19] with fine-tuned Convolutional Neural Networks (CNN) [20], deep learning methods [21] have been applied to aesthetic quality assessment of visual art images, which can automatically learn effective aesthetic features from deep hidden layers to abstract image information without expert knowledge, thus showing outperformed evaluation capability than conventional handcrafted features. Researches in the aesthetic assessment of visual art images using deep learning approaches can be summarized into 4 major schemes (Figure 12), 1) Aesthetic scoring refers to Designs for effective loss functions and structures in output layers of CNNs [83,84] Distribution-oriented aesthetic representation [100] Context-aware attention-based module with distance loss function [85] Employ attributes as middlelevel representations to measure aesthetics [45,86] Jointly learn distributions of attributes and aesthetics [87,88] Employ CNN-LSTM architecture to produce comments on photo aesthetic aspects [46,47,49] Attentive LSTM for generating emotional explanations in artworks [53] different aesthetic quality levels (binary labels as "positive" or "negative"), or continuous images aesthetic ratings; 2) Aesthetic distribution refers to the distribution histogram of aesthetic scores of images; 3) Aesthetic attribute refers to the evaluation of good lighting, color harmony, shallow depth of field, balancing element, motion blur and other aspects of images; 4) Aesthetic description refers to linguistic aesthetics comments of images, as shown in Table 3.…”

“…However, there are many strategies for sharing knowledge and learning process for different problems. For example, Zhang et al [48] share parameters in all layers and learn the common features for all tasks, while Liu et al [49] just sharing in some bottom layers and learn respective representation in some top layers for each task. Yim et al [39] treat all tasks equally important.…”

“…Jin et al [49] proposed Aesthetic Attributes Assessment of Images, which means the aesthetic attributes captioning. This work is the first attempt to produce both captions and scores for each image aesthetic attribute, including color and lighting, composition, depth and focus, impression and subject, use of camera.…”