Improving Object Detection in Art Images Using Only Style Transfer

Kadish, David; Risi, Sebastian; Løvlie, Anders Sundnes

doi:10.1109/ijcnn52387.2021.9534264

Cited by 18 publications

(11 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We notice that our semi-supervised learning technique on People-Art always results in an improvement of Average Precision (AP) and Average Recall (AR). Moreover, AP maintains this advantage as the proportion of style-transferred material increases, but becomes [20] and Gonthier et al [13] for the data set, respectively. For PoPArt, we find that semi-supervised learning with art-historical images enhances AP less; thus, our proposed method with COCO 2017 annotations has similar performance to using style transfer.…”

Section: Ablation Studymentioning

confidence: 90%

Semi-supervised Human Pose Estimation in Art-historical Images

Springstein¹,

Schneider²,

Althaus³

et al. 2022

Preprint

View full text Add to dashboard Cite

Gesture as 'language' of non-verbal communication has been theoretically established since the 17th century. However, its relevance for the visual arts has been expressed only sporadically. This may be primarily due to the sheer overwhelming amount of data that traditionally had to be processed by hand. With the steady progress of digitization, though, a growing number of historical artifacts have been indexed and made available to the public, creating a need for automatic retrieval of art-historical motifs with similar body constellations or poses. Since the domain of art differs significantly from existing real-world data sets for human pose estimation due to its style variance, this presents new challenges. In this paper, we propose a novel approach to estimate human poses in art-historical images. In contrast to previous work that attempts to bridge the domain gap with pre-trained models or through style transfer, we suggest semi-supervised learning for both object and keypoint detection. Furthermore, we introduce a novel domain-specific art data set that includes both bounding box and keypoint annotations of human figures. Our approach achieves significantly better results than methods that use pre-trained models or style transfer.

show abstract

Section: Ablation Studymentioning

confidence: 90%

Semi-supervised Human Pose Estimation in Art-historical Images

Springstein¹,

Schneider²,

Althaus³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Images are reduced to a maximum scale of 1, 333 × 800 pixels without changing the aspect ratio. In contrast to previous studies by Kadish et al [36] and Gonthier et al [30], we include difficult-to-annotate figures. As evident by the benchmark results shown in Table 2, state-of-the-art models such as TOOD [22] and PVT [81] outperform multistage R-CNNs to a nearly similar extent in Average Precision (AP) between 1.7 and 4.8 %.…”

Section: Image Collectionmentioning

confidence: 99%

“…For training and validation, PoPArt was used in addition to People-Art. In contrast to previous benchmarks by Kadish et al[36] and Gonthier et al[30], we include difficult-to-annotate figures. The best performing approach is indicated in bold.…”

mentioning

confidence: 99%

Poses of People in Art: A Data Set for Human Pose Estimation in Digital Art History

Schneider,

Vollmer,

Huber

2023

View full text Add to dashboard Cite

Throughout the history of art, the pose-as the holistic abstraction of the human body's expression-has proven to be a constant in numerous studies. However, due to the enormous amount of data that so far had to be processed by hand, its crucial role to the formulaic recapitulation of art-historical motifs since antiquity could only be highlighted selectively. This is true even for the now automated estimation of human poses, as domain-specific, sufficiently large data sets required for training computational models are either not publicly available or not indexed at a fine enough granularity. With the Poses of People in Art data set, we introduce the first openly licensed data set for estimating human poses in art and validating human pose estimators. It consists of 2,454 images from 22 art-historical depiction styles, including those that have increasingly turned away from lifelike representations of the body since the 19 th century. A total of 10,749 human figures are precisely enclosed by rectangular bounding boxes, with a maximum of four per image labeled by up to 17 keypoints; among these are mainly joints such as elbows and knees. For machine learning purposes, the data set is divided into three subsets-training, validation, and testing-, that follow the established JSON-based Microsoft Common Objects in Context (COCO) format, respectively. Each image annotation, in addition to mandatory fields, provides metadata from the art-historical online encyclopedia WikiArt. With this paper, we elaborate on the acquisition and constitution of the data set, address various application scenarios, and discuss prospects for a digitally supported art history. We show that the data set enables the comprehensive investigation of body phenomena in art, whether at the level of individual figures, which can thus be captured in their subtleties, or entire figure constellations, whose position, distance, or proximity to one another is considered.

show abstract

“…Studies on the style transfer of images using deep neural networks have been actively conducted [32]- [37]. Leon A. Gatys [32] published a paper on style transfer using convolutional neural networks.…”

Section: Learning Content Before Stylementioning

confidence: 99%

“…In some studies, domain adaptation and object detection were attempted using style transfer techniques [36], [37].…”

Section: Learning Content Before Stylementioning

confidence: 99%

Data-Centric Learning Method for Synthetic Data Augmentation and Object Detection

Park¹,

Kim²,

Park³

et al. 2022

JOIV : Int. J. Inform. Visualization

View full text Add to dashboard Cite

This paper proposes a deep learning framework for decreasing large-scale domain shift problems in object detection using domain adaptation techniques. We have approached data-centric domain adaptation with Image-to-Image translation models for this problem. It is one of the methodologies that changes source data to target domain's style by reducing domain shift. However, the method cannot be applied directly to the domain adaptation task because the existing Image-to-Image model focuses on style translation. We solved this problem using the data-centric approach simply by reordering the training sequence of the domain adaptation model. We defined the features to be content and style. We hypothesized that object-specific information in images was more closely tied to the content than the style and thus experimented with methods to preserve content information before style was learned. We trained the model separately only by altering the training data. Our experiments confirmed that the proposed method improves the performance of the domain adaptation model and increases the effectiveness of using the generated synthetic data for training object detection models. We compared our approach with the existing single-stage method where content and style were trained simultaneously. We argue that our proposed method is more practical for training object detection models than others. The emphasis in this study is to preserve image content while changing the style of the image. In the future, we plan to conduct additional experiments to apply synthetic data generation technology to various other application areas like indoor scenes and bin picking.

show abstract

Improving Object Detection in Art Images Using Only Style Transfer

Cited by 18 publications

References 12 publications

Semi-supervised Human Pose Estimation in Art-historical Images

Semi-supervised Human Pose Estimation in Art-historical Images

Poses of People in Art: A Data Set for Human Pose Estimation in Digital Art History

Data-Centric Learning Method for Synthetic Data Augmentation and Object Detection

Contact Info

Product

Resources

About