Exemplar-based depth inpainting with arbitrary-shape patches and cross-modal matching

Xiang, Sen; Hu, Deng; Zhu, Lei; Wu, Jin; Yu, Li

doi:10.1016/j.image.2018.07.005

Cited by 13 publications

(6 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A classical method of image inpainting is patch-based methods [12]- [14]. Energy-based method [15] and reference-based method [16] have also been proposed. Recently, the combination of CNN and GAN has produced promising results in reflecting the context of the target image in the completed image.…”

Section: Generative Image Extension (Examples From Our Results)mentioning

confidence: 99%

Scenery Image Extension via Inpainting With a Mirrored Input

2021

View full text Add to dashboard Cite

Generative image extension has the advantage of extending the overall image size while preserving the target image because, unlike other image extensions using interpolation, it completes the surroundings of the target image. However, existing generative image extension methods tend to have poor quality in the generation of outer pixels. One method deals only with a limited number of scene classes because the extension repeats the same semantics. We propose a mirrored input, which sandwiches the extended region by mirroring a part of the target image. This replaces generative image extension with an image inpainting problem and thus helps to achieve higher quality pixel generation and can extend semantics with more complex shapes than horizontal repetition. Experimental results show that our proposed method achieves a scenery image extension that exceeds the state-of-the-art generative image extension methods in both visual quality and FID score for datasets containing diverse scenes.INDEX TERMS image extension, image generation, image inpainting, generative adversarial networks

show abstract

Section: Generative Image Extension (Examples From Our Results)mentioning

confidence: 99%

Scenery Image Extension via Inpainting With a Mirrored Input

2021

View full text Add to dashboard Cite

show abstract

“…These perplexities have been addressed in Criminisi et al (2004); Wang et al (2013); Xiang et al (2019) using exemplar-based texture synthesis approach presented by Efros and Leung Efros & Leung (1999). These schemes scan the source region Φ to obtain the best-exemplar patches for the region Ω, determined by similarity metric.…”

Section: Image Inpaintingmentioning

confidence: 99%

Crypt-OR:A privacy-preserving system for exemplar-based object-removal over the cloud

Tanwar¹,

Raman²,

Bhargava³

2020

Preprint

View full text Add to dashboard Cite

<div>Object removal is a technique for removing the undesired object(s) and then fill-in the empty region(s) in an image such that the modified image is visually plausible. The existing algorithms are unable to provide promising results when the region to be removed - has varying textured-neighborhood, is small in size and the depth of the image and, is of specific geometric shapes such as triangle</div><div>and rectangle. In this paper, we proposed a new algorithm by incorporating the merits of partial differential equations (PDEs) and exemplar-based schemes to address these challenges. The data term, which measures the continuity of</div><div>isophotes in exemplar-based methods, is modified by incorporating a regularizer term and partial derivatives up to second order of the input image. This regularizer enhances the strength of isophotes striking the boundary and boosts</div><div>the information propagation in an unbiased manner, in terms of pixel intensity values. Additionally, the low-cost, agility, and accessing flexibility benefits of cloud services have attracted user’s attention today. Besides, users are concerned about utilizing them for their data, as they are supported by untrusted third parties. Addressing these privacy concerns for object-removal in an image over the cloud server, we extended and modified our algorithm to make it compatible for (T; N)-threshold Shamir secret sharing scheme (SSS). This privacy-preserving system is an end-to-end system for object-removal in the ED over the cloud server namely Crypt-OR. Crypt-OR is evaluated by removing synthetically imposed objects in real-images. Further, Crypt-OR has proved to be secure under various pixel-based cryptographic attacks such as frequency-known attack and pixel-correlation attack. </div>

show abstract

“…CHO et al [10] estimated the boundaries using color images and converged from the edges to the target boundary, effectively addressing holes between edges and multiple edge pixels to achieve seamless inpainting of object boundaries near the camera through an iterative process of alternating filling directions. Xiang et al [11] extended Criminisi's inpainting approach [12] by incorporating a block matching algorithm for identifying irregularly shaped voids and a hybrid matching algorithm for selecting optimal repair patches with minimal differences in hole regions, leading to successful restoration, particularly for larger irregular holes. Zhang et al [13] introduced an object-oriented segmentation method to segment different objects in color images, locate source regions for repair patches, and utilize a search algorithm to match missing depth information regions with suitable objects, ultimately filling depth information gaps through optimal repair patch selection within the segmented objects.…”

Section: Introductionmentioning

confidence: 99%

Non-Local Means Hole Repair Algorithm Based on Adaptive Block

Zhao,

Li,

Pan

2023

Applied Sciences

View full text Add to dashboard Cite

RGB-D cameras provide depth and color information and are widely used in 3D reconstruction and computer vision. In the majority of existing RGB-D cameras, a considerable portion of depth values is often lost due to severe occlusion or limited camera coverage, thereby adversely impacting the precise localization and three-dimensional reconstruction of objects. In this paper, to address the issue of poor-quality in-depth images captured by RGB-D cameras, a depth image hole repair algorithm based on non-local means is proposed first, leveraging the structural similarities between grayscale and depth images. Second, while considering the cumbersome parameter tuning associated with the non-local means hole repair method for determining the size of structural blocks for depth image hole repair, an intelligent block factor is introduced, which automatically determines the optimal search and repair block sizes for various hole sizes, resulting in the development of an adaptive block-based non-local means algorithm for repairing depth image holes. Furthermore, the proposed algorithm’s performance are evaluated using both the Middlebury stereo matching dataset and a self-constructed RGB-D dataset, with performance assessment being carried out by comparing the algorithm against other methods using five metrics: RMSE, SSIM, PSNR, DE, and ALME. Finally, experimental results unequivocally demonstrate the innovative resolution of the parameter tuning complexity inherent in-depth image hole repair, effectively filling the holes, suppressing noise within depth images, enhancing image quality, and achieving elevated precision and accuracy, as affirmed by the attained results.

show abstract

Exemplar-based depth inpainting with arbitrary-shape patches and cross-modal matching

Cited by 13 publications

References 30 publications

Scenery Image Extension via Inpainting With a Mirrored Input

Scenery Image Extension via Inpainting With a Mirrored Input

Crypt-OR:A privacy-preserving system for exemplar-based object-removal over the cloud

Non-Local Means Hole Repair Algorithm Based on Adaptive Block

Contact Info

Product

Resources

About