Instance-Aware Image Colorization

Su, Jheng-Wei; Chu, Hung‐Kuo; Huang, Jia-Bin

doi:10.1109/cvpr42600.2020.00799

Cited by 181 publications

(258 citation statements)

References 24 publications

Supporting

Mentioning

236

Contrasting

Unclassified

Order By: Relevance

“…Even though colorization became involved with machine learning with the appearance of the example-based category, user intervention remained necessary in providing a priori knowledge of the problem or choosing suitable reference images. Some contemporary methods may be classified in both user-guided and deep learning categories [9,10].…”

Section: Colorization Methodsmentioning

confidence: 99%

“…Different colorization methods work with different color spaces. While some authors analyze the influence of various color spaces in the colorization process [6,7], many choose the convenient one and develop the method with the selected color space [8,9,10,11].…”

Section: Introductionmentioning

confidence: 99%

“…In this paper, a detailed review of the existing colorization methods has been conducted along with the quantitative evaluation of the results. The algorithms of Iizuka et al [6], Zhang et al [8], Levin et al [3], Su et al [10], Vitoria et al [11] and Zhang et al [9] (both automatic and interactive versions) have been analyzed and evaluated. Colorization has been done on five distinctive photographs made by the author.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Grayscale Image Colorization Methods: Overview and Evaluation

et al. 2021

View full text Add to dashboard Cite

Colorization is a process of transforming grayscale images to color images in a visually acceptable way. The main goal is to convince a viewer in the authenticity of the result. Grayscale images requiring colorization are in most cases images with natural scenes. Over the past 20 years a wide range of colorization methods has been developedfrom algorithmically simple, yet time-and energy-consuming because of unavoidable human intervention to more complicated, but simultaneously more automated methods. Automatic conversion has become a challenging area that links machine learning and deep learning with art. This paper presents an overview and evaluation of the grayscale image colorization methods and techniques applied to natural images. The paper provides classification of the existing colorization methods, explains the principles which they are based on and emphasizes their advantages and disadvantages. Special focus is put on deep learning methods. Relevant methods are compared regarding image quality and processing time. Different metrics for colored image quality assessment are used in this paper. Measuring the perceived quality of a colored image is extremely challenging because of the complexity of the human visual system. Multiple metrics used for colorization method evaluation provide results by determining the difference between the predicted color value and the ground-truth, which in several cases is not in coherence with image plausibility. The results indicate that user-guided colorization neural networks are the most promising colorization category because of the successful merge of human intervention and neural network automation. INDEX TERMSAutomatic methods, black-and-white image, colorfulness, colorization, deep learning methods, example-based methods, grayscale image, image quality assessment, scribble-based methods, userguided methods.

show abstract

Section: Colorization Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Grayscale Image Colorization Methods: Overview and Evaluation

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Grayscale images are the input for the system, which outputs the corresponding color images directly, without manual intervention. The fully automatic image colorization systems [3,19,34,42] achieve highquality performance that generates high-quality colorized images owing to the emergence and development of DNN technology. A lot of work has also tried to apply a Generative Adversarial Network (GAN) [9] to colorization, focusing on improving training stability and making robust color image synthesis in large multi-class image datasets [10,15,28,32].…”

Section: Related Workmentioning

confidence: 99%

“…A lot of work has proposed fully automatic colorization [3,10,15,19,28,32,34,42]. Most of it was based on the deep neural network (DNN) [40,41] and achieved high-quality image colorization.…”

Section: Introductionmentioning

confidence: 99%

Unpaired medical image colorization using generative adversarial network

Liang

Lee

et al. 2021

Multimed Tools Appl

View full text Add to dashboard Cite

We consider medical image transformation problems where a grayscale image is transformed into a color image. The colorized medical image should have the same features as the input image because extra synthesized features can increase the possibility of diagnostic errors. In this paper, to secure colorized medical images and improve the quality of synthesized images, as well as to leverage unpaired training image data, a colorization network is proposed based on the cycle generative adversarial network (CycleGAN) model, combining a perceptual loss function and a total variation (TV) loss function. Visual comparisons and experimental indicators from the NRMSE, PSNR, and SSIM metrics are used to evaluate the performance of the proposed method. The experimental results show that GAN-based style conversion can be applied to colorization of medical images. As well, the introduction of perceptual loss and TV loss can improve the quality of images produced as a result of colorization better than the result generated by only using the CycleGAN model.

show abstract

UGSC‐GAN: User‐guided sketch colorization with deep convolution generative adversarial networks

Zhang

Zhu

Liu

et al. 2021

Computer Animation & Virtual

View full text Add to dashboard Cite

Inspired by the creating process of human paintings, we propose a novel adversarial architecture for multiple sketch colorization which is a scribble-based, automatic, and exemplar-based colorization method. The proposed framework has two stages, namely, imitating stage and shading stage. In the imitating stage, to address the challenge of lack of texture in the sketch, we train a grayscale generation network to accomplish a mapping task, namely, generating a grayscale map with textured, grayscale, boundary information from the input sparse sketch. In the shading stage, the model can accurately colorize the objects in the gray image generated in the previous stage, and generate high-quality colorized images. With the proposed model trained on our database, the experimental results show that our method can generate vivid colorized images and achieve a better performance than previous methods evaluated by FID metric.

show abstract

Instance-Aware Image Colorization

Cited by 181 publications

References 24 publications

Grayscale Image Colorization Methods: Overview and Evaluation

Grayscale Image Colorization Methods: Overview and Evaluation

Unpaired medical image colorization using generative adversarial network

UGSC‐GAN: User‐guided sketch colorization with deep convolution generative adversarial networks

Contact Info

Product

Resources

About