How to predict the global instantaneous feeling induced by a facial picture?

Abstract

International audiencePicture selection is a time-consuming task for humans and a real challenge for machines, which have to retrieve complex and subjective information from image pixels. An automated system that infers human feelings from digital portraits would be of great help for profile picture selection, photo album creation or photo editing. In this work, two models of facial pictures evaluation are defined. The first one predicts the overall aesthetic quality of a facial image, and the second one answers the question " Among a set of facial pictures of a given person, on which picture does the person look like the most friendly? ". Aesthetic quality is evaluated by the computation of 15 features that encode low-level statistics in different image regions (face, eyes, mouth). Relevant features are automatically selected by a feature ranking technique, and the outputs of 4 learning algorithms are fused in order to make a robust and accurate prediction of the image quality. Results are compared with recent works and the proposed algorithm obtains the best performance. The same pipeline is considered to evaluate the likability of a facial picture, with the difference that the estimation is based on high-level attributes such as gender, age, smile. Performance of these attributes is compared with previous techniques that mostly rely on facial keypoints positions, and it is shown that it is possible to obtain likability predictions that are close to human perception. Finally, a combination of both models that selects a likable facial image of good quality for a given person is described

A Genetic Algorithm to Combine Deep Features for the Aesthetic Assessment of Images Containing Faces

“…For the experiments, the same evaluation procedure adopted in [ 19 ] was followed. More in detail, for each experiment, ten-fold cross-validation was performed by randomly dividing the dataset into ten disjoint subsets and repeating the experiment ten times, each time selecting a different subset of tests and the remaining nine for training.…”

“…The size of the smallest face in the database is pixels, while the largest face almost completely covers the surface of the image with a size of pixels. According to [ 19 ], only the biggest detected face is considered in each picture. Figure 11 a shows samples from the database, while the distribution of the scores is reported in Figure 11 b.…”

“…Previously proposed methods for the aesthetic quality assessment of images containing faces can be grouped into those that treat the problem as a categorization into images with low or high aesthetic quality [ 18 , 19 , 20 ] and those that instead estimate a continuous score of aesthetic quality [ 1 , 17 , 19 ].…”

“…Li et al [ 21 ] evaluated the performance of several categories of features related to aesthetics such as pose, face locations, and photo composition on their own dataset of photos with faces. Lienhard et al [ 19 , 22 ] proposed a new database, called Human Faces Score (HFS), and developed a method based on the selection of low-level features extracted from several regions for both aesthetic quality categorization of portrait images (i.e., low or high) and continuous aesthetic score prediction. Recently, many works have proposed intelligent capture methods for taking good selfies based on hand-crafted features and face pose analysis [ 17 , 23 ].…”

Facial Image Manipulation

“…Vernon, Sutherland, Young, and Hartley (2014) utilized landmark locations across 1,000 naturalistic face images to define facial features (e.g., eye size) and then used these features to predict facial impressions. This approach gave insights into the pattern of facial cues underlying impressions and allowed impressions to be automatically extracted from new photographs, with obvious applications for real-world impression prediction (Vernon, Sutherland, Young, & Hartley, 2014; see also Lienhard, Ladret, & Caplier, 2015). The approach was inspired by pioneering work in social psychology by Zebrowitz and colleagues who used facial landmarks to test the theory that facial impressions are based on subtle resemblance to emotional expression and to investigate stereotypes (e.g., Zebrowitz, Kikuchi, & Fellous, 2010; Zebrowitz & Montepare, 1992).…”