Visual complexity modelling based on image features fusion of multiple kernels

Abstract

Humans’ perception of visual complexity is often regarded as one of the key principles of aesthetic order, and is intimately related to the physiological, neurological and, possibly, psychological characteristics of the human mind. For these reasons, creating accurate computational models of visual complexity is a demanding task. Building upon on previous work in the field (Forsythe et al., 2011; Machado et al., 2015) we explore the use of Machine Learning techniques to create computational models of visual complexity. For that purpose, we use a dataset composed of 800 visual stimuli divided into five categories, describing each stimulus by 329 features based on edge detection, compression error and Zipf’s law. In an initial stage, a comparative analysis of representative state-of-the-art Machine Learning approaches is performed. Subsequently, we conduct an exhaustive outlier analysis. We analyze the impact of removing the extreme outliers, concluding that Feature Selection Multiple Kernel Learning obtains the best results, yielding an average correlation to humans’ perception of complexity of 0.71 with only twenty-two features. These results outperform the current state-of-the-art, showing the potential of this technique for regression.

Comparison of Outlier-Tolerant Models for Measuring Visual Complexity

“…Neural Networks and, particularly, Multilayer Perceptrons, have to be over-adjusted, especially if a large number of samples are not available [ 49 ]. There are other methods that may be more suitable for prediction tasks and that minimize the problems discussed, such as combining predictors from many different models [ 50 ], such as those used by Fernandez-Lozano et al [ 21 ], or “the drop-out technique”.…”

“…Fernandez-Lozano et al [ 21 ] studied other computational methods in order to identify alternatives to ANNs already used to solve problems related to visual complexity. Until then, the best results obtained had been R-Squared = 0.69 using as an input set 329 features with a multilayer ANN [ 20 ].…”

“…Until then, the best results obtained had been R-Squared = 0.69 using as an input set 329 features with a multilayer ANN [ 20 ]. Among the methods studied, Fernandez-Lozano et al [ 21 ] concluded that the most reliable for the problem raised was FSMKL, which obtained a result of 0.71, emphasizing that such results could be obtained with only 22 metrics. These were repeated 50 times in independent experiments, obtaining an RMSE with little variability.…”

“…This study proposes other models based on Machine Learning that apply to the same input data used by Forsythe et al [ 26 ], Machado et al [ 20 ] and Fernandez-Lozano et al [ 21 ].…”

“…Machado et al (2015) [ 20 ] used the same dataset in their work, using artificial neural networks (ANN) and computer-generated metrics with Machine Learning to predict the visual complexity of images. In 2019, Fernandez-Lozano et al [ 21 ] used the same dataset to assess the value of visual complexity through the application and validation of a set of Machine Learning models. They calculated more than 300 features based on the image compression error and Zipf’s law on three color channels.…”