Sparse pixel sampling for appearance edit propagation

Abstract

Edit propagation is an appearance-editing method using sparsely provided edit strokes from users. Although edit propagation has a wide variety of applications, it is computationally complex, owing to the need to solve large linear systems. To reduce the computational cost, interpolation-based approaches have been studied intensely. This study is inspired by an interpolation-based edit-propagation method that uses a clustering algorithm to determine samples. The method uses an interpolant, which approximates edit parameters with convex combinations of the samples. However, because the clustering algorithm generates samples that lie inside the set of pixels in a feature space, an interpolant with convex combinations does not allow for an exact reconstruction of the pixels outside the convex hull. To address this issue, this paper proposes a novel approximation model for interpolating image colors as well as edit parameters using affine combinations. In addition, this paper introduces sparse pixel sampling to determine the quantity and positions of samples and the weight coefficients of the affine combinations simultaneously. Sparse pixel sampling is performed by updating candidate pixels. Unnecessary pixels are discarded with compressive sensing, and new candidate pixels are greedily resampled following their approximation errors. This paper demonstrates that the B Tatsuya Yatagawa proposed model achieves better approximation in terms of both image colors and edit parameters, and discusses the properties of the proposed model with various experiments.

Efficient propagation of sparse edits on 360∘ panoramas

“…By reformulating edit propagation as a function interpolation problem in a high-dimensional feature space, Li et al [6] efficiently solved the problem using radial basis functions. Another interpolation based method, proposed by Yatagawa et al [11], approximated the edit parameters with convex combinations of samples, which can achieve a better accuracy in terms of colors and edit parameters. Another acceleration approach worth mentioning is the hierarchical data structure based method [12] which achieved scalable edit propagation.…”

DeepProp: Extracting Deep Features from a Single Image for Edit Propagation

“…AppProp [AP08] yields better propagation by optimizing color differences for optimizing color differences not only between nearby pixels, but also between non‐neighboring ones. Computational efficiency has been also improved by using a kd‐tree [XLJ*09], continuously approximating feature space using radial basis functions (RBFs) [LJH10], manifold learning [MCY*13], efficient stroke sampling [BHW11], and sparse pixel sampling [YY15]. Most of the previous approaches share a common issue, namely that halo artifacts occur across object boundaries [LWA*12].…”

“…Many efforts have been made to attack the edit propagation problem [LLW04,LWCO * 07, LAA08, XLJ * 09,XWT * 09,LJH10,CZZT12,XYJ13,CZL * 14, YY15]. Various applications of edit propagation exist, such as grayscale image colorization, color image recoloring, segmentation and tone adjustment.…”

“…Various applications of edit propagation exist, such as grayscale image colorization, color image recoloring, segmentation and tone adjustment. Many efforts have been made to attack the edit propagation problem [LLW04, LWCO*07, LAA08, XLJ*09, XWT*09, LJH10, CZZT12, XYJ13, CZL*14, YY15]. Specifying sparse image edits, users can propagate the image edits to the entire image according to the propagation principle based on pixel similarity (e.g., proximities of positions, colors or textures).…”

Editorial

“…The paper by Debattista et al [32] is about the compression of high dynamic range images. Yatagawa and Yamaguchi [33] present a method for the appearance editing for images. The paper by Hua and Wang [34] presents an image completion method and the last paper related to image processing is by Qiao et al [35]; they describe a technique to generate QR codes that are visually similar to an input image.…”