StyleBabel: Artistic Style Tagging and Captioning

Abstract

We present StyleBabel, a unique open access dataset of natural language captions and free-form tags describing the artistic style of over 135K digital artworks, collected via a novel participatory method from experts studying at specialist art and design schools. StyleBabel was collected via an iterative method, inspired by 'Grounded Theory': a qualitative approach that enables annotation while co-evolving a shared language for fine-grained artistic style attribute description. We demonstrate several downstream tasks for StyleBabel, adapting the recent ALADIN architecture for fine-grained style similarity, to train cross-modal embeddings for: 1) free-form tag generation; 2) natural language description of artistic style; 3) fine-grained text search of style. To do so, we extend ALADIN with recent advances in Visual Transformer (ViT) and cross-modal representation learning, achieving a state of the art accuracy in fine-grained style retrieval.

Exploring Emotional Stimuli Detection in Artworks: A Benchmark Dataset and Baselines Evaluation

“…Artwork analysis Much effort has been dedicated to solving art-related problems with machine learning techniques, including style identification [ 1 , 32 ], object detection [ 1 , 3 , 4 ], instance-level recognition [ 33 ], or artwork description [ 2 , 9 , 34 ]. Concerning emotion analysis, some datasets [ 1 , 11 , 12 , 35 ], including ArtEmis, contain labels with the emotion (e.g., amusement and fear ) that each artwork evokes.…”