PlenoPatch: Patch-Based Plenoptic Image Manipulation

Zhang, Fang-Lue; Wang, Jue; Shechtman, Eli; Zhou, Ziye; Shi, Jiaxin; Hu, Shi‐Min

doi:10.1109/tvcg.2016.2532329

Cited by 78 publications

(40 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In Ref. [24], Zhang et al extended patch-based synthesis to plenoptic images captured by consumer-level lenslet-based devices for interactive light field editing. They represented the light field as a set of images captured from different viewpoints and performed patch-based image synthesis on all affected layers of the central view, and then propagated the edits to all other views.…”

Section: Patch-based Synthesismentioning

confidence: 99%

Image editing by object-aware optimal boundary searching and mixed-domain composition

Jin

et al. 2018

Comp. Visual Media

View full text Add to dashboard Cite

When combining very different images which often contain complex objects and backgrounds, producing consistent compositions is a challenging problem requiring seamless image editing. In this paper, we propose a general approach, called objectaware image editing, to obtain consistency in structure, color, and texture in a unified way. Our approach improves upon previous gradient-domain composition in three ways. Firstly, we introduce an iterative optimization algorithm to minimize mismatches on the boundaries when the target region contains multiple objects of interest. Secondly, we propose a mixeddomain consistency metric for measuring gradients and colors, and formulate composition as a unified minimization problem that can be solved with a sparse linear system. In particular, we encode texture consistency using a patch-based approach without searching and matching. Thirdly, we adopt an objectaware approach to separately manipulate the guidance gradient fields for objects of interest and backgrounds of interest, which facilitates a variety of seamless image editing applications. Our unified method outperforms previous state-of-the-art methods in preserving global texture consistency in addition to local structure continuity.

show abstract

Section: Patch-based Synthesismentioning

confidence: 99%

Image editing by object-aware optimal boundary searching and mixed-domain composition

Jin

et al. 2018

Comp. Visual Media

View full text Add to dashboard Cite

show abstract

“…One way to look at light fields is as an image array with many different camera viewpoints. Zhang et al [53] demonstrated a layered patch-based synthesis system which is designed to manipulate light fields as an image array. This enables users to perform inpainting, and re-arrange and re-layer the content in the light field as shown in Fig.…”

Section: Image Collectionsmentioning

confidence: 99%

“…For large image datasets, Gould and Zhang [56] proposed a method to build a matching graph using PatchMatch, and optimize a conditional Markov random field to propagate pixel labels to all images from just a small subset of annotated Fig. 7 Results of reshuffling and re-layering the content in a light field using PlenoPatch [53]. Reproduced with permission from Ref.…”

Section: Image Collectionsmentioning

confidence: 99%

See 1 more Smart Citation

A survey of the state-of-the-art in patch-based synthesis

Barnes

Zhang

2017

Comp. Visual Media

Self Cite

View full text Add to dashboard Cite

This paper surveys the state-of-the-art of research in patch-based synthesis. Patch-based methods synthesize output images by copying small regions from exemplar imagery. This line of research originated from an area called "texture synthesis", which focused on creating regular or semi-regular textures from small exemplars. However, more recently, much research has focused on synthesis of larger and more diverse imagery, such as photos, photo collections, videos, and light fields. Additionally, recent research has focused on customizing the synthesis process for particular problem domains, such as synthesizing artistic or decorative brushes, synthesis of rich materials, and synthesis for 3D fabrication. This report investigates recent papers that follow these themes, with a particular emphasis on papers published since 2009, when the last survey in this area was published. This survey can serve as a tutorial for readers who are not yet familiar with these topics, as well as provide comparisons between these papers, and highlight some open problems in this area.

show abstract

“…Using an estimated depth map they observe only a few differences on the capacity of the users to perform the editing. Assuming that light field data can be well approximated by a fixed number of scene layers at different depth, a depth-layer-aware image synthesis method is proposed in [14] for edit propagation.…”

Section: Introductionmentioning

confidence: 99%

Superrays for Efficient Light Field Processing

Hog

Sabater

Guillemot

2017

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

Abstract-Light field acquisition devices allow capturing scenes with unmatched post-processing possibilities. However, the huge amount of high dimensional data poses challenging problems to light field processing in interactive time. In order to enable light field processing with a tractable complexity, in this paper, we address the problem of light field over-segmentation. We introduce the concept of super-ray, which is a grouping of rays within and across views, as a key component of a light field processing pipeline. The proposed approach is simple, fast, accurate, easily parallelisable, and does not need a dense depth estimation. We demonstrate experimentally the efficiency of the proposed approach on real and synthetic datasets, for sparsely and densely sampled light fields. As super-rays capture a coarse scene geometry information, we also present how they can be used for real time light field segmentation and correcting refocusing angular aliasing.

show abstract

PlenoPatch: Patch-Based Plenoptic Image Manipulation

Cited by 78 publications

References 62 publications

Image editing by object-aware optimal boundary searching and mixed-domain composition

Image editing by object-aware optimal boundary searching and mixed-domain composition

A survey of the state-of-the-art in patch-based synthesis

Superrays for Efficient Light Field Processing

Contact Info

Product

Resources

About