Insights from a Large-Scale Database of Material Depictions in Paintings

Lin, Hubert; Zuijlen, Mitchell van; Wijntjes, Maarten; Pont, Sylvia C.; Bala, Kavita

doi:10.1007/978-3-030-68796-0_38

Cited by 8 publications

(5 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent work has shown that such neural style transfer algorithms can also produce images that are useful for training robust neural networks [101]. In a related paper, we more explicitly discuss specific applications of the MIP dataset for computer vision [102]. Similarly, other domains of computer vision research might benefit from painterly depictions.…”

Section: Discussionmentioning

confidence: 99%

Materials In Paintings (MIP): An interdisciplinary dataset for perception, art history, and computer vision

et al. 2021

Self Cite

View full text Add to dashboard Cite

In this paper, we capture and explore the painterly depictions of materials to enable the study of depiction and perception of materials through the artists’ eye. We annotated a dataset of 19k paintings with 200k+ bounding boxes from which polygon segments were automatically extracted. Each bounding box was assigned a coarse material label (e.g., fabric) and half was also assigned a fine-grained label (e.g., velvety, silky). The dataset in its entirety is available for browsing and downloading at materialsinpaintings.tudelft.nl. We demonstrate the cross-disciplinary utility of our dataset by presenting novel findings across human perception, art history and, computer vision. Our experiments include a demonstration of how painters create convincing depictions using a stylized approach. We further provide an analysis of the spatial and probabilistic distributions of materials depicted in paintings, in which we for example show that strong patterns exists for material presence and location. Furthermore, we demonstrate how paintings could be used to build more robust computer vision classifiers by learning a more perceptually relevant feature representation. Additionally, we demonstrate that training classifiers on paintings could be used to uncover hidden perceptual cues by visualizing the features used by the classifiers. We conclude that our dataset of painterly material depictions is a rich source for gaining insights into the depiction and perception of materials across multiple disciplines and hope that the release of this dataset will drive multidisciplinary research.

show abstract

Section: Discussionmentioning

confidence: 99%

Materials In Paintings (MIP): An interdisciplinary dataset for perception, art history, and computer vision

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…Today's scientists believe that the development of artificial intelligence and commuter learning based on computational aesthetics as well as developing neuron algorithms will drastically change our daily lives [24,25]. The same goes for problems associated with the field of visual arts and the creative process in generalthese could be resolved with aid of artificial intelligence, which will considerably aid in understanding the artistic legacy of artists and designers of the 21st century [26], including the visual approaches used by artists in different historical periods [27,28]. The newly available computerised tools designed to assess beauty and create aesthetically pleasing objects are being investigated in the framework of computational aesthetics, an emerging interdisciplinary field of inquiry which functions at the fringes of science and art.…”

Section: Discussionmentioning

confidence: 99%

Developing an Approach to Colour Assessment of Works of Art on Aim to Creating a Comfortable and Harmonious Interior

Pylypchuk¹,

Polubok²,

Krivenko³

et al. 2021

Advances in Social Science, Education and Humanities Research

View full text Add to dashboard Cite

This study was specially tailored to develop an approach to colour scheme evaluation for works of art within an interior with aim to create a comfortable and harmonious living space. To achieve the study objectives, the following methods were applied: article review; systematic review of key compositional and colour links which exist between the interior and an art object; algorithmic modelling. The study has resulted in developing standards of coordinating the interior with works of art, namely: the principles of coordinating the interior colour scheme and the visual art objects; the principle of impact of lighting on perception of colour combinations present in art-objects and the interior; the principle of impact of specific surface materials on perception of colour in an art-object an interior. Based on these standards of typical colour combinations in compositional scenarios were developed and analysed, also they were applied in designing flow charts and developing an algorithmic system. The algorithm underwent preliminary testing. The obtained results are in favour of its practical application in assessing the colour of art-objects with aim to harmonising the overall colour scheme in an interior.

show abstract

“…A variation of this approach could involve mixing images that differ in ground truth category, but which produce the same kind of material-scale ambiguity (e.g., the images of water and leather in Figure 7B). An intriguing alternative approach would be to ask artists to create ambiguous images and analyze the techniques they use to depict material appearance (Di Cicco et al, 2021;Lin et al, 2021). Such methods may make it possible to identify key features that alter perceived scale or material appearance.…”

Section: Limitations Of the Studymentioning

confidence: 99%