Example-Based Image Colorization Using Locality Consistent Sparse Representation

Abstract

Image colorization aims to produce a natural looking color image from a given gray-scale image, which remains a challenging problem. In this paper, we propose a novel example-based image colorization method exploiting a new locality consistent sparse representation. Given a single reference color image, our method automatically colorizes the target gray-scale image by sparse pursuit. For efficiency and robustness, our method operates at the superpixel level. We extract low-level intensity features, mid-level texture features, and high-level semantic features for each superpixel, which are then concatenated to form its descriptor. The collection of feature vectors for all the superpixels from the reference image composes the dictionary. We formulate colorization of target superpixels as a dictionary-based sparse reconstruction problem. Inspired by the observation that superpixels with similar spatial location and/or feature representation are likely to match spatially close regions from the reference image, we further introduce a locality promoting regularization term into the energy formulation, which substantially improves the matching consistency and subsequent colorization results. Target superpixels are colorized based on the chrominance information from the dominant reference superpixels. Finally, to further improve coherence while preserving sharpness, we develop a new edge-preserving filter for chrominance channels with the guidance from the target gray-scale image. To the best of our knowledge, this is the first work on sparse pursuit image colorization from single reference images. Experimental results demonstrate that our colorization method outperforms the state-of-the-art methods, both visually and quantitatively using a user study.

SCGAN: Saliency Map-Guided Colorization With Generative Adversarial Network

“…This idea was enhanced by characterizing the image patches using GMM [27], discriminating different regions by segmentation maps [25], predicting probability for each pixel by global optimization [22], and modelling color selection by energy-minimization method [18]. Moreover, superpixels [19], [21], [24] were utilized to model the correspondences between grayscale input and reference. To alleviate effort of selecting proper reference images, Chia et al [29] developed an image retrieval method to download appropriate reference images from Internet.…”

Fast Hue-Division-Based Selective Color Transfer

“…where, F: →R 3 is the recolored image, consisting of F H as the hue component and (13). By this truncated function, the hue information of the original color will be updated to be similar to the expected color by modifying its circular mean value and rescaling its standard variance value.…”

Automatic Example-Based Image Colorization Using Location-Aware Cross-Scale Matching

“…To improve texture matching, especially near edges, Arbelot et al [18] developed a colour transfer and colourisation method that utilises spatial coherence around image structure by adopting an edge-aware texture descriptor based on region covariance, although local matching is still performed independently. A locality consistent sparse representation learning method is proposed by [19]. By incorporating locality consistency in the matching stage rather than in post-processing as existing methods do, it substantially improves colour consistency and reduces artefacts.…”