Learning the Principles of Art History with convolutional neural networks

Cetinić, Eva; Lipić, Tomislav; Grgić, Sonja

doi:10.1016/j.patrec.2019.11.008

Cited by 34 publications

(25 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To sum up, these studies show that AI is currently widely applied in the field of art, and research areas are gradually expanding. Direct quantification and detection of VDP with AI has not yet been performed, except in a study by Cetinic et al [29] which aims to quantify stylistic properties and to predict the values of Wölfflin's visual principles.…”

Section: Ai-based Studies and Motivations In Artmentioning

confidence: 99%

“…Since we had created the synthetic datasets by defined rules, annotation was not necessary. For other domains, as in the work of Cetinic et al [29], we sought professional support to evaluate the sub-VDP in the images. We consulted a team of five professional architects and five artists, who had equivalent experience in the visual design field.…”

Section: Annotation Of the Datasetmentioning

confidence: 99%

See 1 more Smart Citation

Detecting visual design principles in art and architecture through deep convolutional neural networks

Demir

Cekmis

Yesilkaynak

et al. 2021

Automation in Construction

View full text Add to dashboard Cite

Visual design is associated with the use of some basic design elements and principles. Those are applied by the designers in the various disciplines for aesthetic purposes, relying on an intuitive and subjective process. Thus, numerical analysis of design visuals and disclosure of the aesthetic value embedded in them are considered as hard. However, it has become possible with emerging artificial intelligence technologies. This research aims at a neural network model, which recognizes and classifies the design principles over different domains. The domains include artwork produced since the late 20th century; professional photos; and facade pictures of contemporary buildings. The data collection and curation processes, including the production of computationally-based synthetic dataset, is genuine. The proposed model learns from the knowledge of myriads of original designs, by capturing the underlying shared patterns. It is expected to consolidate design processes by providing an aesthetic evaluation of the visual compositions with objectivity.

show abstract

Section: Ai-based Studies and Motivations In Artmentioning

confidence: 99%

Section: Annotation Of the Datasetmentioning

confidence: 99%

Detecting visual design principles in art and architecture through deep convolutional neural networks

Demir

Cekmis

Yesilkaynak

et al. 2021

Automation in Construction

View full text Add to dashboard Cite

show abstract

“…One of the most important concepts related to DL is transfer learning [1,[18][19][20][21][22][23][24][25][26][27][28][29][30]. Popular programming and software development platforms such as Matlab or Python offer a wide range of pre-trained CNN models of different structures and complexity.…”

Section: Related Workmentioning

confidence: 99%

Adversarial Learning Approach to Unsupervised Labeling of Fine Art Paintings

2021

View full text Add to dashboard Cite

An automatic classification of fine art images is limited by the scarcity of high-quality labels made by art experts. This study aims to provide meaningful automatic labeling of fine art paintings (machine labeling) without the need for human annotation. A new unsupervised Adversarial Clustering System (ACS) is proposed. The ACS is an adversarial learning approach comprising an unsupervised clustering module generating machine labels and a supervised classification module classifying the data based on the machine labels. Both modules are linked through an optimization algorithm iteratively improving the unsupervised clusters. The objective function driving the improvement consists of the within-cluster sum of squares (WCSS) error and the supervised classification accuracy. The proposed method was tested on three different fine-art datasets, including two sets of paintings previously categorized by art experts and one never categorized collection of Australian Aboriginal paintings. The unsupervised clusters were analyzed using standard unsupervised clustering metrics and a reliability measure between machine and human labeling. The ACS showed higher reliability compared to the classical k-means clustering method. The content analysis of unsupervised clusters indicated grouping based on scene composition, type, and shape of the object, edge sharpness and direction, and color palette.

show abstract

“…Scholars who have spent years analyzing and learning the specifics and nuances of fine art can easily identify metadata associated with a painting [ 1 ]; however, identifying metadata is difficult for general audiences who lack expertise. As deep learning has developed, more works have focused on using learned features to conduct art classification [ 2 , 3 , 4 , 5 , 6 , 7 ]. In other words, by training computers using paintings previously labeled by human experts, the machines can learn the image features and classify labels for the images automatically.…”

Section: Introductionmentioning

confidence: 99%

How to Represent Paintings: A Painting Classification Using Artistic Comments

Zhao

Zhou

Qiu

et al. 2021

Sensors

View full text Add to dashboard Cite

The goal of large-scale automatic paintings analysis is to classify and retrieve images using machine learning techniques. The traditional methods use computer vision techniques on paintings to enable computers to represent the art content. In this work, we propose using a graph convolutional network and artistic comments rather than the painting color to classify type, school, timeframe and author of the paintings by implementing natural language processing (NLP) techniques. First, we build a single artistic comment graph based on co-occurrence relations and document word relations and then train an art graph convolutional network (ArtGCN) on the entire corpus. The nodes, which include the words and documents in the topological graph are initialized using a one-hot representation; then, the embeddings are learned jointly for both words and documents, supervised by the known-class training labels of the paintings. Through extensive experiments on different classification tasks using different input sources, we demonstrate that the proposed methods achieve state-of-art performance. In addition, ArtGCN can learn word and painting embeddings, and we find that they have a major role in describing the labels and retrieval paintings, respectively.

show abstract

Learning the Principles of Art History with convolutional neural networks

Cited by 34 publications

References 6 publications

Detecting visual design principles in art and architecture through deep convolutional neural networks

Detecting visual design principles in art and architecture through deep convolutional neural networks

Adversarial Learning Approach to Unsupervised Labeling of Fine Art Paintings

How to Represent Paintings: A Painting Classification Using Artistic Comments

Contact Info

Product

Resources

About