Compare the performance of the models in art classification

Zhao, Wentao; Zhou, Dalin; Qiu, Xinguo; Jiang, Wei

doi:10.1371/journal.pone.0248414

Cited by 19 publications

(14 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ResNet is the dominant CNN architecture. Traditional CNN architectures have been proven to be extremely powerful for visual arts analysis obtaining state-of-the-art performance in style classification [7,15] and artist attribution tasks [62]. We observe a similar effect as in [15].…”

Section: Fine Art Categorizationsupporting

confidence: 78%

Graph Neural Networks for Knowledge Enhanced Visual Representation of Paintings

Efthymiou

Rudinac

Kackovic

et al. 2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

We propose ArtSAGENet, a novel multimodal architecture that integrates Graph Neural Networks (GNNs) and Convolutional Neural Networks (CNNs), to jointly learn visual and semantic-based artistic representations. First, we illustrate the significant advantages of multi-task learning for fine art analysis and argue that it is conceptually a much more appropriate setting in the fine art domain than the single-task alternatives. We further demonstrate that several GNN architectures can outperform strong CNN baselines in a range of fine art analysis tasks, such as style classification, artist attribution, creation period estimation, and tag prediction, while training them requires an order of magnitude less computational time and only a small amount of labeled data. Finally, through extensive experimentation we show that our proposed ArtSAGENet captures and encodes valuable relational dependencies between the artists and the artworks, surpassing the performance of traditional methods that rely solely on the analysis of visual content. Our findings underline a great potential of integrating visual content and semantics for fine art analysis and curation. CCS CONCEPTS• Computing methodologies → Neural networks; Image representations; Multi-task learning; Semantic networks; • Applied computing → Fine arts.

show abstract

Section: Fine Art Categorizationsupporting

confidence: 78%

Graph Neural Networks for Knowledge Enhanced Visual Representation of Paintings

Efthymiou

Rudinac

Kackovic

et al. 2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

show abstract

“…Various categorization tasks related to images 37 can achieve greater performance with datasets of a limited size with transfer learning than using any other method. Previous work has shown that effective performance can be achieved through pre-trained models fine-tuned on specific tasks 38 , 39 .…”

Section: Related Workmentioning

confidence: 99%

Deep learning for COVID-19 detection based on CT images

Zhao

Jiang

Qiu

2021

Sci Rep

Self Cite

130

View full text Add to dashboard Cite

COVID-19 has tremendously impacted patients and medical systems globally. Computed tomography images can effectively complement the reverse transcription-polymerase chain reaction testing. This study adopted a convolutional neural network for COVID-19 testing. We examined the performance of different pre-trained models on CT testing and identified that larger, out-of-field datasets boost the testing power of the models. This suggests that a priori knowledge of the models from out-of-field training is also applicable to CT images. The proposed transfer learning approach proves to be more successful than the current approaches described in literature. We believe that our approach has achieved the state-of-the-art performance in identification thus far. Based on experiments with randomly sampled training datasets, the results reveal a satisfactory performance by our model. We investigated the relevant visual characteristics of the CT images used by the model; these may assist clinical doctors in manual screening.

show abstract

“…The performed analyses span several classification tasks and techniques: from style classification to artist identification, comprising also medium, school, and year classification [ 27 , 28 , 29 ]. These researches are useful to support cultural heritage studies and asset management, e.g., automatic cataloguing of unlabeled works in online and museum collections, but their results can be exploited for more complex applications, such as authentication, stylometry [ 30 ], and forgery detection [ 31 ].…”

Section: Related Workmentioning

confidence: 99%

Comparing CAM Algorithms for the Identification of Salient Image Features in Iconography Artwork Analysis

et al. 2021

View full text Add to dashboard Cite

Iconography studies the visual content of artworks by considering the themes portrayed in them and their representation. Computer Vision has been used to identify iconographic subjects in paintings and Convolutional Neural Networks enabled the effective classification of characters in Christian art paintings. However, it still has to be demonstrated if the classification results obtained by CNNs rely on the same iconographic properties that human experts exploit when studying iconography and if the architecture of a classifier trained on whole artwork images can be exploited to support the much harder task of object detection. A suitable approach for exposing the process of classification by neural models relies on Class Activation Maps, which emphasize the areas of an image contributing the most to the classification. This work compares state-of-the-art algorithms (CAM, Grad-CAM, Grad-CAM++, and Smooth Grad-CAM++) in terms of their capacity of identifying the iconographic attributes that determine the classification of characters in Christian art paintings. Quantitative and qualitative analyses show that Grad-CAM, Grad-CAM++, and Smooth Grad-CAM++ have similar performances while CAM has lower efficacy. Smooth Grad-CAM++ isolates multiple disconnected image regions that identify small iconographic symbols well. Grad-CAM produces wider and more contiguous areas that cover large iconographic symbols better. The salient image areas computed by the CAM algorithms have been used to estimate object-level bounding boxes and a quantitative analysis shows that the boxes estimated with Grad-CAM reach 55% average IoU, 61% GT-known localization and 31% mAP. The obtained results are a step towards the computer-aided study of the variations of iconographic elements positioning and mutual relations in artworks and open the way to the automatic creation of bounding boxes for training detectors of iconographic symbols in Christian art images.

show abstract

Compare the performance of the models in art classification

Cited by 19 publications

References 36 publications

Graph Neural Networks for Knowledge Enhanced Visual Representation of Paintings

Graph Neural Networks for Knowledge Enhanced Visual Representation of Paintings

Deep learning for COVID-19 detection based on CT images

Comparing CAM Algorithms for the Identification of Salient Image Features in Iconography Artwork Analysis

Contact Info

Product

Resources

About