A Closed-Form Solution to Photorealistic Image Stylization

Abstract

Photorealistic image stylization concerns transferring style of a reference photo to a content photo with the constraint that the stylized photo should remain photorealistic. While several photorealistic image stylization methods exist, they tend to generate spatially inconsistent stylizations with noticeable artifacts. In this paper, we propose a method to address these issues. The proposed method consists of a stylization step and a smoothing step. While the stylization step transfers the style of the reference photo to the content photo, the smoothing step ensures spatially consistent stylizations. Each of the steps has a closedform solution and can be computed efficiently. We conduct extensive experimental validations. The results show that the proposed method generates photorealistic stylization outputs that are more preferred by human subjects as compared to those by the competing methods while running much faster. Source code and additional results are available at https://github.com/NVIDIA/FastPhotoStyle.

End-To-End Time-Lapse Video Synthesis From a Single Outdoor Image

“…As the result, 60.6% of our results were perceived as real images by the users. In contrast, the corresponding numbers for real images, Li et al's method [19], and Shih et al's method [27] are 67.5%, 34.1%, and 44.9%, respectively. Our percentage is also higher than the reported value in [27], which is 55.2%.…”

“…In contrast, our results were mostly preferred by the users without using any reference video. show frames from a retrieved reference video, the results from [19], [27], and our method, respectively.…”

Semantic Attribute Matching Networks

“…To alleviate these limitations, Luan et al proposed a deep photo style transfer [38] that computes and uses the semantic labels. Li et al proposed Photo-WCT [32] to eliminate the artifacts using additional smoothing step. However, these methods still have been formulated without considering semantically meaningful correspondence fields.…”

“…Effects on the confidence in neural patch blending: (a) blending results of I s and I t , (b) blending results of F s and F t followed by the decoder, (c) confidence, and (d) blending results of F s and F t with the confidence followed by the decoder. and high-level attribute characteristics [31,32,12]. However, using the skip connection through simple concatenation [44] makes the decoder networks reconstruct an image using only low-level features.…”

Learning to Train with Synthetic Humans

“…However, GAN-based methods are prone to unstable training and require tedious hyper-parameter tuning. As a practical alternative, recently [10] proposed domain stylization to stylize synthetic images to real ones, using the fast photorealistic image stylization method of [25].…”