Colorization of Line Drawings with Empty Pupils

Akita, Kenta; Morimoto, Yasuhiko; Tsuruno, Reiji

doi:10.1111/cgf.14171

Cited by 10 publications

(10 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In Ref. [13], the authors proposed a learning-based network model that transfers pupil details from a reference colour image to an empty pupil image. And there are also some methods to calculate the colour statistics of the input image and the reference image, and then establish a mapping function to map the colour distribution of the reference image to the input image, such as the colour transfer techniques [14][15][16].…”

Section: Automatic Colourisation Methods Based On the Reference Imagementioning

confidence: 99%

LineGAN: An image colourisation method combined with a line art network

Nie

2022

IET Computer Vision

View full text Add to dashboard Cite

The work on grayscale image colourisation has been significantly improved. Currently, learning-based methods have achieved some great colourisation effects, but existing colour edge bleeding, especially when colourful cartoon characters. In this paper, we focus on the colourisation of cartoon characters from a series in an adversarial environment with a line art network, whose name is LineGAN. LineGAN learns the corresponding colour mapping from datasets, improving the accuracy of image colourisation. Our methods limit the colour boundary overflow by adding a line art frame in the generator. Extensive experiment results on cartoon image colourisation tasks demonstrate that the proposed method can achieve effective results.

show abstract

Section: Automatic Colourisation Methods Based On the Reference Imagementioning

confidence: 99%

LineGAN: An image colourisation method combined with a line art network

Nie

2022

IET Computer Vision

View full text Add to dashboard Cite

show abstract

“…The latter is a very challenging case, which needs model require a high precision extraction of local features and semantic correspondence. To the best of our knowledge, we are the first learning based work can generate results with accurate color in pupils according to the reference image with no extra eyes segmentation label [8] or pupil position estimation network [52].…”

Section: Qualitative Evaluationmentioning

confidence: 99%

AnimeDiffusion: Anime Face Line Drawing Colorization via Diffusion Models

Cao¹,

Meng²,

Mok³

et al. 2023

Preprint

View full text Add to dashboard Cite

It is a time-consuming and tedious work for manually colorizing anime line drawing images, which is an essential stage in cartoon animation creation pipeline. Reference-based line drawing colorization is a challenging task that relies on the precise cross-domain long-range dependency modelling between the line drawing and reference image. Existing learning methods still utilize generative adversarial networks (GANs) as one key module of their model architecture. In this paper, we propose a novel method called AnimeDiffusion using diffusion models that performs anime face line drawing colorization automatically. To the best of our knowledge, this is the first diffusion model tailored for anime content creation. In order to solve the huge training consumption problem of diffusion models, we design a hybrid training strategy, first pre-training a diffusion model with classifier-free guidance and then fine-tuning it with image reconstruction guidance. We find that with a few iterations of fine-tuning, the model shows wonderful colorization performance, as illustrated in Fig. 1. For training AnimeDiffusion, we conduct an anime face line drawing colorization benchmark dataset, which contains 31696 training data and 579 testing data. We hope this dataset can fill the gap of no available high resolution anime face dataset for colorization method evaluation. Through multiple quantitative metrics evaluated on our dataset and a user study, we demonstrate AnimeDiffusion outperforms state-of-the-art GANs-based models for anime face line drawing colorization. We also collaborate with professional artists to test and apply our AnimeDiffusion for their creation work. We release our code on https://github.com/xqmeng/AnimeDiffusion.

show abstract

“…However, traditional methods [SDC09, PMC22, SMYA14, FTR18] are usually developed on the basis of user‐guided hints, so they are inappropriate for reference‐based colourization. As neural networks have been proven effective in object recognition and colour rendering [ZIE16, ZZI*17, XHZ*20], many deep learning models have been proposed to solve the sketch colourization problem by encoding text [HYES19, ZMG*19, CMG21], user‐guided hints [FHOO17, ZLW*18] and reference images [ZJLL17, LKL*20, SLWW19, AMT20] into colour information. Among the reference‐based methods, Akita et al.…”

Section: Related Workmentioning

confidence: 99%

“…Among the reference‐based methods, Akita et al. proposed a method [AMT20] that can fill empty pupils with reference images, but their method is only effective for eyes in portraits; Style2paints [ZLW*18] can generate satisfactory results by introducing a pre‐trained Inception network [CLJ*15], but the colours in their results differ from the reference image and cannot be edited using tags. Inspired by Style2paints, we investigate how a pre‐trained encoder benefits reference‐based colourization and propose the second training to enable tag‐based manipulation.…”

Section: Related Workmentioning

confidence: 99%

Two‐Step Training: Adjustable Sketch Colourization via Reference Image and Text Tag

Yan

Ito

Moriai

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

Automatic sketch colourization is a highly interestinged topic in the image‐generation field. However, due to the absence of texture in sketch images and the lack of training data, existing reference‐based methods are ineffective in generating visually pleasant results and cannot edit the colours using text tags. Thus, this paper presents a conditional generative adversarial network (cGAN)‐based architecture with a pre‐trained convolutional neural network (CNN), reference‐based channel‐wise attention (RBCA) and self‐adaptive multi‐layer perceptron (MLP) to tackle this problem. We propose two‐step training and spatial latent manipulation to achieve high‐quality and colour‐adjustable results using reference images and text tags. The superiority of our approach in reference‐based colourization is demonstrated through qualitative/quantitative comparisons and user studies with existing network‐based methods. We also validate the controllability of the proposed model and discuss the details of our latent manipulation on the basis of experimental results of multi‐label manipulation.

show abstract

Colorization of Line Drawings with Empty Pupils

Abstract: The input line drawing (left), color reference image (middle), and our result (right). Our method can paint details in empty pupils by transferring pupil details from the reference image.

Cited by 10 publications

References 23 publications

LineGAN: An image colourisation method combined with a line art network

LineGAN: An image colourisation method combined with a line art network

AnimeDiffusion: Anime Face Line Drawing Colorization via Diffusion Models

Two‐Step Training: Adjustable Sketch Colourization via Reference Image and Text Tag

Contact Info

Product

Resources

About