Parametric meta-filter modeling from a single example pair

Huang, Shuo; Zhang, Guoxin; Lai, Yuekun; Kopf, Johannes; Cohen–Or, Daniel; Hu, Shi‐Min

doi:10.1007/s00371-014-0973-y

Cited by 11 publications

(10 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Instead of developing specific NPR algorithms which require substantial effort for each style, style transfer has been actively researched. Unlike traditional style transfer methods [11,12] which require paired style/non-style images, recent studies [19,1,7,8] show that the VGG network [30] trained for object recognition has good ability to extract semantic features of objects, which is very important in stylization. As a result, more powerful style transfer methods have been developed which do not require paired training images.…”

Section: Stylization With Neural Networkmentioning

confidence: 99%

CartoonGAN: Generative Adversarial Networks for Photo Cartoonization

Yang

Lai

Liu

2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

Self Cite

331

211

View full text Add to dashboard Cite

In this paper, we propose a solution to transforming photos of real-world scenes into cartoon style images, which is valuable and challenging in computer vision and computer graphics. Our solution belongs to learning based methods, which have recently become popular to stylize images in artistic forms such as painting. However, existing methods do not produce satisfactory results for cartoonization, due to the fact that (1) cartoon styles have unique characteristics with high level simplification and abstraction, and (2) cartoon images tend to have clear edges, smooth color shading and relatively simple textures, which exhibit significant challenges for texture-descriptor-based loss functions used in existing methods. In this paper, we propose Car-toonGAN, a generative adversarial network (GAN) framework for cartoon stylization. Our method takes unpaired photos and cartoon images for training, which is easy to use. Two novel losses suitable for cartoonization are proposed: (1) a semantic content loss, which is formulated as a sparse regularization in the high-level feature maps of the VGG network to cope with substantial style variation between photos and cartoons, and (2) an edge-promoting adversarial loss for preserving clear edges. We further introduce an initialization phase, to improve the convergence of the network to the target manifold. Our method is also much more efficient to train than existing methods. Experimental results show that our method is able to generate high-quality cartoon images from real-world photos (i.e., following specific artists' styles and with clear edges and smooth shading) and outperforms state-of-the-art methods.

show abstract

Section: Stylization With Neural Networkmentioning

confidence: 99%

CartoonGAN: Generative Adversarial Networks for Photo Cartoonization

Yang

Lai

Liu

2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

Self Cite

331

211

View full text Add to dashboard Cite

show abstract

“…Compared with image filtering [20], it is a more challenging topic since extending image-based filtering algorithms to video sequences is nontrivial, and is especially difficult for low-quality underexposed videos we are targeting. Lee et al [21] constructed a spatio-temporal anisotropic diffusion framework to remove noise by a weighted average on the spatiotemporal pixels.…”

Section: Related Workmentioning

confidence: 99%

Underexposed Video Enhancement via Perception-Driven Progressive Fusion

Zhang

Nie

Zhang

et al. 2016

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

Underexposed video enhancement aims at revealing hidden details that are barely noticeable in LDR video frames with noise. Previous work typically relies on a single heuristic tone mapping curve to expand the dynamic range, which inevitably leads to uneven exposure and visual artifacts. In this paper, we present a novel approach for underexposed video enhancement using an efficient perception-driven progressive fusion. For an input underexposed video, we first remap each video frame using a series of tentative tone mapping curves to generate an multi-exposure image sequence that contains different exposed versions of the original video frame. Guided by some visual perception quality measures encoding the desirable exposed appearance, we locate all the best exposed regions from multi-exposure image sequences and then integrate them into a well-exposed video in a temporally consistent manner. Finally, we further perform an effective texture-preserving spatio-temporal filtering on this well-exposed video to obtain a highquality noise-free result. Experimental results have shown that the enhanced video exhibits uniform exposure, brings out noticeable details, preserves temporal coherence, and avoids visual artifacts. Besides, we demonstrate applications of our approach to a set of problems including video dehazing, video denoising and HDR video reconstruction. he visited University of California-Davis for one year.His main interests include image and video processing, digital geometry processing, and computational photography. He is a member of IEEE.

show abstract

“…The Identity filter is the corresponding channel of image A, which passes through the channel unchanged. Channel Offset filter provides a constant channel offset for each corresponding channel like in Huang et al (2014). In our method, we set the constants of the three Channel Offset filters to 0.01.…”

Section: Basis Filters Selectionmentioning

confidence: 99%

“…Kaiming et al (2013) construct a linear combination of local mappings within windows of the guided images to approximate the input image. Recently, Huang et al (2014) propose a model which is the state-of-the-art method to learn an image filter from a single pair of input images. They model a compound filter by a linear combination of basis filters from filter bank.…”

Section: Introductionmentioning

confidence: 99%

Laplace operator based multi-channel image filters learning

Liu

et al. 2016

JAMDSM

View full text Add to dashboard Cite

Learning an image filter from a pair of images composed of an original image and its filtered version has attracted increasing attention in the image processing region. However, the existing methods only take the color information into consideration to learn an image filter, which cannot accurately depict the luminance and some style changes of images so that the generated learning results are inaccurate. This paper proposes a Laplace operator based multichannel learning approach to learn a multi-channel filter from an original image and its filtered version. Different from previous method, we adopt a strategy to learn a multi-channel filter which can accurately reconstruct the filtered image. In addition, benefiting from the Laplace regularization based multi-channel learning model, we can make the neighbor pixels of each channel have the similar weights and learn more homogeneous weight maps for each channel filter, thereby generating more accurate results with less time. Due to the multi-channel representation of our learnt filter, we can edit the luminance style and the color style of images by changing the weights of different channel filter. Furthermore, we can edit the effect of the image filters by tuning the weights of some basis filters. At last, extensive experiments well validate the performance of our method over state-of-the-art methods in terms of accuracy and speed.

show abstract

Parametric meta-filter modeling from a single example pair

Cited by 11 publications

References 32 publications

CartoonGAN: Generative Adversarial Networks for Photo Cartoonization

CartoonGAN: Generative Adversarial Networks for Photo Cartoonization

Underexposed Video Enhancement via Perception-Driven Progressive Fusion

Laplace operator based multi-channel image filters learning

Contact Info

Product

Resources

About