Classification of basic artistic media based on a deep convolutional approach

Yang, Heekyung; Min, Kyungha

doi:10.1007/s00371-019-01641-6

Cited by 24 publications

(9 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This work demonstrates that art type prediction is more accurate using the fine-tuned AlexNet convolutional neural network (CNN) architecture than when implementing a support vector machine (SVM) with SIFT features [24]. Yang and Min [39] perform classification on multiple art datasets using various CNN architectures and confirm that DenseNet [12] achieves the best performance. Unlike the dataset used in [39] which contains mostly paintings, our dataset consists of various artwork types.…”

Section: Related Work 21 Multi-class and Multi-label Classificationmentioning

confidence: 79%

“…Yang and Min [39] perform classification on multiple art datasets using various CNN architectures and confirm that DenseNet [12] achieves the best performance. Unlike the dataset used in [39] which contains mostly paintings, our dataset consists of various artwork types. Therefore, in this paper we have chosen ResNet [9] as backbone neural architecture which is proved to obtain good results on multiple computer vision tasks, e.g., object detection and image classification.…”

Section: Related Work 21 Multi-class and Multi-label Classificationmentioning

confidence: 89%

“…The application of artificial intelligence techniques to the cultural heritage field has attracted increasing attention in recent years [8,21,22,[28][29][30]39]. Most of these work focus on automatic metadata annotation such as predicting the author, material, and date of an artwork.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Generating Captions for Images of Ancient Artworks

Sheng

Moens

2019

Proceedings of the 27th ACM International Conference on Multimedia

View full text Add to dashboard Cite

The neural encoder-decoder framework is widely adopted for image captioning of natural images. However, few works have contributed to generating captions for cultural images using this scheme. In this paper, we propose an artwork type enriched image captioning model where the encoder represents an input artwork image as a 512-dimensional vector and the decoder generates a corresponding caption based on the input image vector. The artwork type is first predicted by a convolutional neural network classifier and then merged into the decoder. We investigate multiple approaches to integrate the artwork type into the captioning model among which is one that applies a step-wise weighted sum of the artwork type vector and the hidden representation vector of the decoder. This model outperforms three baseline image captioning models for a Chinese art image captioning dataset on all evaluation metrics. One of the baselines is a state-of-the-art approach fusing textual image attributes into the captioning model for natural images. The proposed model also obtains promising results for another Egyptian art image captioning dataset. CCS CONCEPTS • Information systems → Multimedia content creation; • Computing methodologies → Scene understanding.

show abstract

Section: Related Work 21 Multi-class and Multi-label Classificationmentioning

confidence: 79%

Section: Related Work 21 Multi-class and Multi-label Classificationmentioning

confidence: 89%

See 1 more Smart Citation

Generating Captions for Images of Ancient Artworks

Sheng

Moens

2019

Proceedings of the 27th ACM International Conference on Multimedia

View full text Add to dashboard Cite

show abstract

“…One of the most important concepts related to DL is transfer learning [1,[18][19][20][21][22][23][24][25][26][27][28][29][30]. Popular programming and software development platforms such as Matlab or Python offer a wide range of pre-trained CNN models of different structures and complexity.…”

Section: Related Workmentioning

confidence: 99%

Adversarial Learning Approach to Unsupervised Labeling of Fine Art Paintings

2021

View full text Add to dashboard Cite

An automatic classification of fine art images is limited by the scarcity of high-quality labels made by art experts. This study aims to provide meaningful automatic labeling of fine art paintings (machine labeling) without the need for human annotation. A new unsupervised Adversarial Clustering System (ACS) is proposed. The ACS is an adversarial learning approach comprising an unsupervised clustering module generating machine labels and a supervised classification module classifying the data based on the machine labels. Both modules are linked through an optimization algorithm iteratively improving the unsupervised clusters. The objective function driving the improvement consists of the within-cluster sum of squares (WCSS) error and the supervised classification accuracy. The proposed method was tested on three different fine-art datasets, including two sets of paintings previously categorized by art experts and one never categorized collection of Australian Aboriginal paintings. The unsupervised clusters were analyzed using standard unsupervised clustering metrics and a reliability measure between machine and human labeling. The ACS showed higher reliability compared to the classical k-means clustering method. The content analysis of unsupervised clusters indicated grouping based on scene composition, type, and shape of the object, edge sharpness and direction, and color palette.

show abstract

“…We follow this strategy for our recognizer. Recently, we have tested five widely-used structures, including AlexNet, VGGNet, GoogLeNet, ResNet, and DenseNet, for recognizing artistic media from real artwork images and concluded that DenseNet, the latest CNN structure, shows best performance among them [18]. Therefore, we employ DenseNet-161 [19] for our recognizer.…”

Section: A Strategy For Our Recognizermentioning

confidence: 99%

A Multi-Column Deep Framework for Recognizing Artistic Media

Yang

Min

2019

Electronics

View full text Add to dashboard Cite

We present a multi-column structured framework for recognizing artistic media from artwork images. We design the column of our framework using a deep neural network. Our key idea is to recognize the distinctive stroke texture of an artistic medium, which plays a key role in distinguishing artistic media. Since stroke texture is in a local scale, the whole image is not proper for recognizing the texture. Therefore, we devise two ideas for our framework: Sampling patches from an input image and employing a Gram matrix to extract the texture. The patches sampled from an input artwork image are processed in the columns of our framework to make local decisions on the patch, and the local decisions from the patches are merged to make a final decision for the input artwork image. Furthermore, we employ a Gram matrix, which is known to effectively capture texture information, to improve the accuracy of recognition. Our framework is trained and tested using two real artwork image datasets: WikiSet of traditional artwork images and YMSet of contemporary artwork images. Finally, we build SynthSet, which is a collection of synthesized artwork images from many computer graphics literature, and propose a guideline for evaluating the synthesized artwork images.

show abstract

Classification of basic artistic media based on a deep convolutional approach

Cited by 24 publications

References 33 publications

Generating Captions for Images of Ancient Artworks

Generating Captions for Images of Ancient Artworks

Adversarial Learning Approach to Unsupervised Labeling of Fine Art Paintings

A Multi-Column Deep Framework for Recognizing Artistic Media

Contact Info

Product

Resources

About