Smart Culture Lens: An Application That Analyzes the Visual Elements of Ceramics

Abstract

The Smart Culture Lens developed in this study is an application developed by utilizing the visual element classification system of ceramic and AI technology. The user can analyze the visual elements of the ceramic photo taken with a smartphone and search for similar ceramics related to each visual element. For this development, as a first step, visual elements such as color, form, material, and pattern were defined as criteria for classifying ceramic appearance, and the visual element classification system of ceramics was organized. In the second step, 19,610 images of 7,346 ceramics were collected through museum visit photography and web, and a database was built by annotating these images with a visual element classification system. In the third step, representative object detection models, Faster R-CNN and Mask R-CNN were trained based on a ceramic classification system. Through those trained object detection models, visual elements and masks of the input image were recognized, and representative colors of the area were extracted using the k-means algorithm through the recognized masks. The performance of the trained object detection models (Average precision of Form / Material / Pattern 1st-level category = 0.87 / 0.89 / 0.72) shows that the amount of collected data and the established classification system are useful. Finally, by applying the above development results, a mobile application called 'Smart Culture Lens' was developed, and the usefulness of this application was confirmed through a user experience test. This study combines AI technology into cultural heritage so that people can intuitively explore artifacts from a new perspective, which differs from traditional artifact exploring methods. All the detailed processes of this development will be a guide to how to apply AI technology to the cultural heritage.INDEX TERMS Classification scheme of ceramics, visual elements analysis, visual search, ceramic dataset. I. INTRODUCTION

An analysis of the appearance characteristics of Korean ceramics per era through statistical analysis of metadata annotated with a visual element classification system of ceramics

“…When data annotating artifacts' visual element classification system is put to good use, objective explorations and statistical interpretations of artifacts become possible. In fact, by converging AI technology on the basis of ceramic classification system annotation data, the present researchers developed the Smart Culture Lens application, which automatically analyzes the visual elements of photographs of surrounding ceramics taken by the user [1]. This app was favorably evaluated by users as allowing them to obtain information on artifacts from an intuitive perspective.…”

“…As for the white porcelain of Joseon, Fig. 5 Form classification categories of Korean earthenware and ceramics [1] the Bunwon supplying products to the royal household was managed and came to lead the production of quality white porcelain. At this production center, ceramics were produced by mixing white porcelain clay collected from across the nation and quality, high-purity kaolin, and the firing temperatures rose considerably.…”

“…As shown in Fig. 1, the Smart Culture Lens [1] that applied AI technology to the data of Korean traditional ceramics developed by this research team used visual search technology such as Google Lens (https:// lens. google.…”

“…This Smart Culture Lens uses computer vision and machine learning technology to automatically recognize visual elements, such as shape, material, pattern, and color of ceramics, when one ceramic is photographed with a smartphone in a museum [1]. In addition, it provides related visual element information analyzed through automatic analysis technology, as well as the results and displays of ceramics with the same visual elements for each element through search technology.…”

Ceramic Microscope Image Classification Based on Multi-Scale Fusion Bottleneck Structure and Chunking Attention Mechanism

“…Jiapeng et al [7] used neural networks to classify ceramic images of different macroscopic shapes and achieved an accuracy of 92.62%. Yi et al [8] constructed a set of ceramic classification standards for visual elements such as shape, color, and pattern of ceramics and achieved 72% pattern classification accuracy through target detection by using neural networks, ultimately formed a ceramic classification system. Chetouani et al [9], [10] automatically classified the ceramic fragment images by constructing a Convolutional neural network and achieved the best accuracy.…”