Visual Link Retrieval in a Database of Paintings

Séguin, Benoît; Striolo, Carlotta; diLenardo, Isabella; Kaplan, Frédé́ric

doi:10.1007/978-3-319-46604-0_52

Cited by 64 publications

(64 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Unsurprisingly, the pre-trained models obtained the worst results among all the baselines, as they do not present enough discriminative power in the domain of art. Also, there was a clear improvement with respect to pre-trained baselines when the networks were fine-tuned, as already noted in previous work [30,39,44,45]. On the other hand, adding attributes or captions to the visual representations seemed to improve the accuracy, although not in all the scenarios, e.g., Timeframe was better classified with the fine-tuned ResNet50 model than with ResNet50+Attributes or ResNet50+Captions, whereas School was better classified with the fine-tuned ResNet50 than with ResNet50+Captions.…”

Section: Results Analysissupporting

confidence: 80%

See 1 more Smart Citation

Context-Aware Embeddings for Automatic Art Analysis

García

Renoust

Nakashima

2019

Proceedings of the 2019 on International Conference on Multimedia Retrieval

View full text Add to dashboard Cite

Automatic art analysis aims to classify and retrieve artistic representations from a collection of images by using computer vision and machine learning techniques. In this work, we propose to enhance visual representations from neural networks with contextual artistic information. Whereas visual representations are able to capture information about the content and the style of an artwork, our proposed context-aware embeddings additionally encode relationships between different artistic attributes, such as author, school, or historical period. We design two different approaches for using context in automatic art analysis. In the first one, contextual data is obtained through a multi-task learning model, in which several attributes are trained together to find visual relationships between elements. In the second approach, context is obtained through an art-specific knowledge graph, which encodes relationships between artistic attributes. An exhaustive evaluation of both of our models in several art analysis problems, such as author identification, type classification, or cross-modal retrieval, show that performance is improved by up to 7.3% in art classification and 37.24% in retrieval when context-aware embeddings are used.

show abstract

Section: Results Analysissupporting

confidence: 80%

“…in which computer vision techniques are applied to study the content [12,39], the style [9,38], or to classify the attributes [30,32] of a specific piece of art.…”

Section: Introductionmentioning

confidence: 99%

Context-Aware Embeddings for Automatic Art Analysis

García

Renoust

Nakashima

2019

Proceedings of the 2019 on International Conference on Multimedia Retrieval

View full text Add to dashboard Cite

show abstract

“…Our goal, however, is to go further and focus on the computational analysis of relationships between individual artworks. Seguin et al [40,39] propose to find visual relationships in collections of paintings. However, while they use off-the-shelf CNNs trained in a supervised manner, we focus on the design of a new self-supervised feature learning specifically trained for the task.…”

Section: Computer Vision and Artmentioning

confidence: 99%

Discovering Visual Patterns in Art Collections With Spatially-Consistent Feature Learning

Shen

Efros

Aubry

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Our goal in this paper is to discover near duplicate patterns in large collections of artworks. This is harder than standard instance mining due to differences in the artistic media (oil, pastel, drawing, etc), and imperfections inherent in the copying process. Our key technical insight is to adapt a standard deep feature to this task by fine-tuning it on the specific art collection using self-supervised learning. More specifically, spatial consistency between neighbouring feature matches is used as supervisory fine-tuning signal. The adapted feature leads to more accurate style-invariant matching, and can be used with a standard discovery approach, based on geometric verification, to identify duplicate patterns in the dataset. The approach is evaluated on several different datasets and shows surprisingly good qualitative discovery results. For quantitative evaluation of the method, we annotated 273 near duplicate details in a dataset of 1587 artworks attributed to Jan Brueghel and his workshop 1 . Beyond artworks, we also demonstrate improvement on localization on the Oxford5K photo dataset as well as on historical photograph localization on the Large Time Lags Location (LTLL) dataset.

show abstract

“…Modern image retrieval methods, both local feature and CNN based, have been applied to search in datasets of artworks [3], [4]. This approach can be further improved with human-in-the-loop, as demonstrated by [5] where the annotations are iteratively refined and the CNN model is retrained.…”

Section: Prior Workmentioning

confidence: 99%

Linking Art through Human Poses

Jenícek

Chum

2019

2019 International Conference on Document Analysis and Recognition (ICDAR)

View full text Add to dashboard Cite

We address the discovery of composition transfer in artworks based on their visual content. Automated analysis of large art collections, which are growing as a result of art digitization among museums and galleries, is an important tool for art history and assists cultural heritage preservation. Modern image retrieval systems offer good performance on visually similar artworks, but fail in the cases of more abstract composition transfer. The proposed approach links artworks through a pose similarity of human figures depicted in images. Human figures are the subject of a large fraction of visual art from middle ages to modernity and their distinctive poses were often a source of inspiration among artists. The method consists of two steps -fast pose matching and robust spatial verification. We experimentally show that explicit human pose matching is superior to standard content-based image retrieval methods on a manually annotated art composition transfer dataset.

show abstract

Visual Link Retrieval in a Database of Paintings

Cited by 64 publications

References 24 publications

Context-Aware Embeddings for Automatic Art Analysis

Context-Aware Embeddings for Automatic Art Analysis

Discovering Visual Patterns in Art Collections With Spatially-Consistent Feature Learning

Linking Art through Human Poses

Contact Info

Product

Resources

About