ChromaGAN: Adversarial Picture Colorization with Semantic Class Distribution

Vitoria, Patricia; Raad, Lara; Ballester, Coloma

doi:10.1109/wacv45572.2020.9093389

Cited by 136 publications

(173 citation statements)

References 25 publications

Supporting

Mentioning

162

Contrasting

Order By: Relevance

“…Thus, their colorization effects are dif-ferent, e.g. classifying color for pixels [37] promotes very colorful results; training with scene classification [38], [44] ensures overall color correctness; contextual loss [52], [77] facilitates color similarity with ground truth. Moreover, to alleviate color bleeding and semantic confusion, additional constrains such as gradient loss [31], segmentation loss [34], [35], and bilateral loss [43] were proposed.…”

Section: Related Workmentioning

confidence: 99%

“…However, the problems of color bleeding and unreasonable assignment of colors still exist. Figure 1 shows some common examples of the failure cases of [37] and [44] on some legacy photos. For instance, there is color bleeding in the first column by methods [37], [44] since color of trees spreads to crowd.…”

Section: Introductionmentioning

confidence: 99%

“…Figure 1 shows some common examples of the failure cases of [37] and [44] on some legacy photos. For instance, there is color bleeding in the first column by methods [37], [44] since color of trees spreads to crowd. Also, the roads and human faces are colorized in blue (in column 4 and 5, respectively) by methods [37], [44].…”

Section: Introductionmentioning

confidence: 99%

“…For instance, there is color bleeding in the first column by methods [37], [44] since color of trees spreads to crowd. Also, the roads and human faces are colorized in blue (in column 4 and 5, respectively) by methods [37], [44]. [37], [44] and proposed method on old black and white photographs.…”

Section: Introductionmentioning

confidence: 99%

“…Also, the roads and human faces are colorized in blue (in column 4 and 5, respectively) by methods [37], [44]. [37], [44] and proposed method on old black and white photographs. The rows from top to bottom represent grayscale input, colorization results of [37], [44] and proposed method and saliency map generated by proposed method, respectively.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

SCGAN: Saliency Map-Guided Colorization With Generative Adversarial Network

Zhao

Cheung

et al. 2021

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

Given a grayscale photograph, the colorization system estimates a visually plausible colorful image. Conventional methods often use semantics to colorize grayscale images. However, in these methods, only classification semantic information is embedded, resulting in semantic confusion and color bleeding in the final colorized image. To address these issues, we propose a fully automatic Saliency Map-guided Colorization with Generative Adversarial Network (SCGAN) framework. It jointly predicts the colorization and saliency map to minimize semantic confusion and color bleeding in the colorized image. Since the global features from pre-trained VGG-16-Gray network are embedded to the colorization encoder, the proposed SCGAN can be trained with much less data than state-of-the-art methods to achieve perceptually reasonable colorization. In addition, we propose a novel saliency map-based guidance method. Branches of the colorization decoder are used to predict the saliency map as a proxy target. Moreover, two hierarchical discriminators are utilized for the generated colorization and saliency map, respectively, in order to strengthen visual perception performance. The proposed system is evaluated on ImageNet validation set. Experimental results show that SCGAN can generate more reasonable colorized images than state-of-the-art techniques.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

SCGAN: Saliency Map-Guided Colorization With Generative Adversarial Network

Zhao

Cheung

et al. 2021

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

show abstract

Subjective evaluation of colourized images with different colorization models

Xiao

Liu

et al. 2020

Color Research & Application

View full text Add to dashboard Cite

Two psychophysical experiments were conducted to evaluate the performance of grayscale image colorization models, and to verify the objective image quality metrics adopted in grayscale image colorization. Twenty representative grayscale images were colourized by four colorization models and three typical metrics, root mean square error (RMSE), peak signal‐to‐noise ratio (PSNR) and structural similarity index (SSIM), were used to characterize the objective quality of the colourized images. Forty observers were asked to evaluate those images based on their subjective preference in a pair‐comparison experiment, and to evaluate the perceived similarity between the generated and reference colour images using a seven‐point rating scale. The experimental results indicate that different colorization models and objective metrics exhibit different performance in different scenarios. Each colorization method has its own advantages and disadvantages while none of the tested models performed well for all images. For preference, the model proposed by Iizuka et al based on ImageNet performed better while for perceived similarity the models proposed by Zhang et al and Iizuka et al, also based on ImageNet, outperformed the models of Larsson et al and Iizuka et al which were based on the Places dataset. Due to the fact that many objects have instances of distinct colour, a colorization algorithm cannot correctly reconstruct ground truth image for most gray level images, although it was found that perceived similarity and preference ratings of observers were correlated. In addition, it was found that the tested objective metrics correlated poorly with the subjective judgments of the human observers and their performance varied significantly with image content. These findings demonstrate the limitations of current image colorization studies, and it is suggested that due consideration must be given to human visual perception when evaluating the performance of colorization models.

show abstract

Real‐time coloring method of laser surgery video based on generative adversarial network

Pan¹,

Lian

et al. 2023

Int J Imaging Syst Tech

View full text Add to dashboard Cite

Lasers have become an important technology for the medical treatment of cancer. However, the high power of visible laser knives makes conventional endoscopic imaging light pollution severe. This phenomenon can cause the physician to be unable to determine the condition of the lesion site. In this article, a colorization method for laser surgery videos is proposed. The method performs real-time colorization of the laser surgery video under black-andwhite imaging. The main contribution of this work is to propose a new idea for solving laser contamination in medical surgery videos, along with real-time generative adversarial networks (RTGANs). The network takes advantage of generative adversarial networks and residual networks for fast colorization of black and white images. To evaluate the method, we conducted experiments using three datasets. The experimental results show that the RTGAN network takes only 0.026 s to colorize a single frame image. Coloring speed up to 67 times faster than advanced methods, and the Peak Signal to Noise Ratio reached a maximum of 35.736.

show abstract

ChromaGAN: Adversarial Picture Colorization with Semantic Class Distribution

Cited by 136 publications

References 25 publications

SCGAN: Saliency Map-Guided Colorization With Generative Adversarial Network

SCGAN: Saliency Map-Guided Colorization With Generative Adversarial Network

Subjective evaluation of colourized images with different colorization models

Real‐time coloring method of laser surgery video based on generative adversarial network

Contact Info

Product

Resources

About