Digital Comics Image Indexing Based on Deep Learning

Nguyen, Nhu Van; Rigaud, Christophe; Burie, Jean-Christophe

doi:10.3390/jimaging4070089

Cited by 41 publications

(21 citation statements)

References 67 publications

(150 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Four different object types have been detected in [27]. There are also studies on specialized network for comic face detection [28,29] or comic character detection [30].…”

Section: Related Workmentioning

confidence: 99%

Feature visualization in comic artist classification using deep neural networks

Kim

2019

J Big Data

View full text Add to dashboard Cite

Recent progress in computer vision has facilitated the scientific understanding of artistic visual features in artworks. Artistic style classification and style transfer are two notable examples of this type of analysis. The former aims to classify artworks into one of the predefined classes. The class type can represent the artist, genre, or painting style that effectively represents the aesthetic features of the artwork [1]. The latter aims to migrate a style from one image to another [2, 3]. This models a reference image's statistical features, which are then used to transform other images. This high-level understanding of visual features enables the effective retrieval, processing, and management of artworks. Both examples have been based on machine learning techniques in recent studies, and deep neural networks in particular. However, there is a noticeable limit in current applications, in that most existing approaches deal with fine arts. Popular art forms, such as comics, have been somewhat overlooked in this trend. Considering the present influence of popular art forms, investigating the distinguishing aspects of different types of popular artworks would be useful.

show abstract

“…Four different object types have been detected in [27]. There are also studies on specialized network for comic face detection [28,29] or comic character detection [30].…”

Section: Related Workmentioning

confidence: 99%

Feature visualization in comic artist classification using deep neural networks

Kim

2019

J Big Data

View full text Add to dashboard Cite

show abstract

“…Another subset of the DCM was fully annotated by humans. Nguyen, Rigaud, and Burie () created DCM_772, a subset of 772 pages from the Digital Comic Museum stratified by publisher. Ground truth is available in the form of object bounding boxes around characters, which are further differentiated into four classes: human‐like, object‐like, animal‐like, and extra (supporting role characters), making this data set potentially suitable for differentiated character detection and recognition.…”

Section: Data Setsmentioning

confidence: 99%

“…Nguyen et al () used an off‐the‐shelf YOLOv2 region proposal network (Redmon & Farhadi, ) with some adjusted priors to detect bounding boxes around panels, characters, and faces. For panels, they achieved comparable or slightly better performance than feature engineering techniques, especially when the testing material was from the same set of comics books as the training material.…”

Section: Analysis Of Visual Structurementioning

confidence: 99%

“…Two recently proposed models based on CNN features (Nguyen et al, ; Dubray & Laubrock, ) achieve significantly better pixel‐based speech balloon segmentation than those based on feature engineering. Additionally, Nguyen et al () obtained good results for speech balloon detection on the eBDthèque by using an off‐the‐shelf DeepLab semantic segmentation network (Chen et al, ).…”

Section: Analysis Of Visual Structurementioning

confidence: 99%

See 1 more Smart Citation

Computational Approaches to Comics Analysis

Laubrock

Dunst

2019

Topics in Cognitive Science

View full text Add to dashboard Cite

Comics are complex documents whose reception engages cognitive processes such as scene perception, language processing, and narrative understanding. Possibly because of their complexity, they have rarely been studied in cognitive science. Modeling the stimulus ideally requires a formal description, which can be provided by feature descriptors from computer vision and computational linguistics. With a focus on document analysis, here we review work on the computational modeling of comics. We argue that the development of modern feature descriptors based on deep learning techniques has made sufficient progress to allow the investigation of complex material such as comics for reception studies, including experimentation and computational modeling of cognitive processes.

show abstract

“…The ever-popular Convolutional Neural Networks (CNN) and its derivations are often used in these said usages as they show great potential in dealing with images. The researches carried out in [20], [21] and [22] through the usage of the object detection models of YOLOv2 [23], a customised Faster R-CNN [24] model and Mask R-CNN [25] respectively, stand as a testimony for this fact.…”

Section: Introductionmentioning

confidence: 99%

Extraction of Semantic Content and Styles in Comic Books

Lenadora¹,

Ranathunge²,

Samarawickrama³

et al. 2020

Int J on Adv. in ICT for Emerging Countries

View full text Add to dashboard Cite

Digitisation of comic books would play a crucial role in identifying new areas in which digital comics can be used. Currently, existing systems in this domain lack the capacity to achieve complete digitisation. Digitisation requires a thorough analysis of the semantic content within comic books. This can be further sub-categorised as detection and identification of comic book characters, extraction and analysis of panels as well as texts, derivation of associations between characters and speech balloons, and analysis of different styles of reading. This paper provides an overview of using several object-detection models to detect semantic content in comics. This analysis showed that, under the constraint of limited computational capacity, YOLOv3 was the best-suited model out of the models evaluated. A study of text extraction and recognition using Optical Character Recognition, a method for determining associable speech balloons, as well as a distance-based approach for associations between characters and speech balloons are also presented here. This association method provides an increased accuracy compared to the Euclidean distance-based approach. Finally, a study on comic style is provided along with a learning model with an accuracy of 0.89 to analyse the reading order of comics.

show abstract

Digital Comics Image Indexing Based on Deep Learning

Cited by 41 publications

References 67 publications

Feature visualization in comic artist classification using deep neural networks

Feature visualization in comic artist classification using deep neural networks

Computational Approaches to Comics Analysis

Extraction of Semantic Content and Styles in Comic Books

Contact Info

Product

Resources

About