Style retrieval from natural images

Tseng, Ting-En; Chang, Wei-Ju; Chen, Chu-Song; Wang, Yu-Chiang Frank

doi:10.1109/icassp.2016.7471939

Cited by 8 publications

(2 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They also provided baseline results of a four-task classification challenge using SIFT features and Fisher vector encoding. The authors of [10] combined HoG, SIFT, GIST and Color Features to classify painting genres, while the authors of [22] proposed a ranking-based method to classify paintings using Random Forests. The authors of [23] presented a histogram-based combination of local and global features for the classification of paintings.…”

Section: Feature Extraction Approachesmentioning

confidence: 99%

Leveraging Known Data for Missing Label Prediction in Cultural Heritage Context

2018

View full text Add to dashboard Cite

Cultural heritage represents a reliable medium for history and knowledge transfer. Cultural heritage assets are often exhibited in museums and heritage sites all over the world. However, many assets are poorly labeled, which decreases their historical value. If an asset’s history is lost, its historical value is also lost. The classification and annotation of overlooked or incomplete cultural assets increase their historical value and allows the discovery of various types of historical links. In this paper, we tackle the challenge of automatically classifying and annotating cultural heritage assets using their visual features as well as the metadata available at hand. Traditional approaches mainly rely only on image data and machine-learning-based techniques to predict missing labels. Often, visual data are not the only information available at hand. In this paper, we present a novel multimodal classification approach for cultural heritage assets that relies on a multitask neural network where a convolutional neural network (CNN) is designed for visual feature learning and a regular neural network is used for textual feature learning. These networks are merged and trained using a shared loss. The combined networks rely on both image and textual features to achieve better asset classification. Initial tests related to painting assets showed that our approach performs better than traditional CNNs that only rely on images as input.

show abstract

Section: Feature Extraction Approachesmentioning

confidence: 99%

Leveraging Known Data for Missing Label Prediction in Cultural Heritage Context

2018

View full text Add to dashboard Cite

show abstract

“…Agarwal et al , 2015, used a combination of SIFT, HOG, GIST and color features to perform genre classification for fine art paintings. Tseng et al , 2016, presented a random forest method for paintings’ style identification and proposed a combination of local and global features for art classification. Other techniques tried to experiment with feature engineering such as brushstroke features to analyze fine art.…”

Section: Introductionmentioning

confidence: 99%

A machine learning framework for enhancing digital experiences in cultural heritage

Belhi

Bouras

Al‐Ali

et al. 2020

JEIM

View full text Add to dashboard Cite

PurposeDigital tools have been used to document cultural heritage with high-quality imaging and metadata. However, some of the historical assets are totally or partially unlabeled and some are physically damaged, which decreases their attractiveness and induces loss of value. This paper introduces a new framework that aims at tackling the cultural data enrichment challenge using machine learning.Design/methodology/approachThis framework focuses on the automatic annotation and metadata completion through new deep learning classification and annotation methods. It also addresses issues related to physically damaged heritage objects through a new image reconstruction approach based on supervised and unsupervised learning.FindingsThe authors evaluate approaches on a data set of cultural objects collected from various cultural institutions around the world. For annotation and classification part of this study, the authors proposed and implemented a hierarchical multimodal classifier that improves the quality of annotation and increases the accuracy of the model, thanks to the introduction of multitask multimodal learning. Regarding cultural data visual reconstruction, the proposed clustering-based method, which combines supervised and unsupervised learning is found to yield better quality completion than existing inpainting frameworks.Originality/valueThis research work is original in sense that it proposes new approaches for the cultural data enrichment, and to the authors’ knowledge, none of the existing enrichment approaches focus on providing an integrated framework based on machine learning to solve current challenges in cultural heritage. These challenges, which are identified by the authors are related to metadata annotation and visual reconstruction.

show abstract