Speech balloon and speaker association for comics and manga understanding

Abstract

Comics and manga are one of the most important forms of publication and play a major role in spreading culture all over the world. In this paper we focus on balloons and their association to comic characters or more generally text and graphic links retrieval. This information is not directly encoded in the image, whether scanned or digital-born, it has to be understood according to other information present in the image. Such high level information allows new browsing experience and story understanding (e.g. dialog analysis, situation retrieval). We propose a speech balloon and comic character association method able to retrieve which character is emitting which speech balloon. The proposed method is based on geometric graph analysis and anchor point selection. This work has been evaluated over various comic book styles from the eBDtheque dataset and also a volume of the Kingdom manga series.

Digital Comics Image Indexing Based on Deep Learning

“…At such level of precision, it is also possible to localize and find the direction of the tail. We proposed a method for this task in 2015 [5] and also to associate speech balloons and comic characters [35].…”

“…We propose to use our previously published methods for detecting the tail position along the edge of the speech balloons, which determines its direction and associates it with a speaker (comic character) [5,35].…”

“…Once we find the tail position, we compute the orientation and direction of the last part of the tail which is directed towards the speaking character. The association with the speaker is established using our method [35] which performed well on 93.32% of the tails from the eBDtheque dataset [88]. In this approach, we build a geometric graph in a Euclidean plane within the panel where vertices are spatial positions of tail and comic character body (or faces if detected) centroids.…”

Text-Independent Speech Balloon Segmentation for Comics and Manga

“…Both information are key issue for balloon classification [1] and comic character association [2]. This last information is not implicitly put by the cartoonist into the drawing but understood by the reader according to the position of the elements in the images.…”

“…Balloon extraction attracted little attention even-though it is an helpful information for text extraction and essential for text/graphics association. lot for generic methods designed for such complex graphics extraction [2]. A pixel-level speech balloon extraction appear to be important for further processing such as balloon contour and tail analysis Fig.…”