Image Inpainting using Orthogonal Viewpoints and Structure Consistency in Manhattan World

Sari, Irawati Nurmala; Urano, Yuto; Du, Weiwei

doi:10.1145/3468081.3471129

Cited by 7 publications

(7 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Researchers have navigated challenges common to both, necessitating robust generalization capabilities for effective image completion. A spectrum of techniques has been explored, from deep neural networks [4,[6][7][8][9][10][11][12] to traditional methods [13][14][15][16][17][18][19][20][21][22][23][24][25].…”

Section: Related Workmentioning

confidence: 99%

Weighted Similarity-Confidence Laplacian Synthesis for High-Resolution Art Painting Completion

Sari,

2024

Preprint

View full text Add to dashboard Cite

Artistic image completion is fundamental for the preservation and restoration of invaluable art paintings and it has experienced significant progress through the implementation of deep learning methodologies. Despite these advancements, challenges persist, even with sophisticated approaches like Generative Adversarial Networks (GANs), particularly in achieving optimal results for high-resolution paintings. Small-scale texture synthesis and the inference of missing information from distant contexts present persistent issues, leading to distortions in lines and unnatural colors, especially in art paintings with complicated structures and textures. Concurrently, patch-based image synthesis has evolved by incorporating global optimization on the image pyramid to enhance structural coherence and details. However, methods relying on gradient-based synthesis encounter obstacles related to directionality, inconsistency, and the heavy computational burdens associated with solving the Poisson equation in non-integrable gradient fields. This paper introduces a groundbreaking approach, integrating Weighted Similarity-Confidence Laplacian Synthesis, to comprehensively address these challenges and advance the field of artistic image completion. This proposal addresses challenges not only in high-resolution artistic image completion but also makes a significant contribution to the broader field of patch-based synthesis by utilizing the Laplacian pyramid for enhanced edge-aware correspondence search. Experimental result confirms the effectiveness of our approach, offering promising outcomes for the preservation and restoration of art paintings with complicated details and irregular missing regions.

show abstract

Section: Related Workmentioning

confidence: 99%

Weighted Similarity-Confidence Laplacian Synthesis for High-Resolution Art Painting Completion

Sari,

2024

Preprint

View full text Add to dashboard Cite

show abstract

Section: Related Workmentioning

confidence: 99%

Weighted Similarity-Confidence Laplacian Synthesis for High-Resolution Art Painting Completion

Sari,

2024

Applied Sciences

View full text Add to dashboard Cite

Artistic image completion assumes a significant role in the preservation and restoration of invaluable art paintings, marking notable advancements through the adoption of deep learning methodologies. Despite progress, challenges persist, particularly in achieving optimal results for high-resolution paintings. The intricacies of complex structures and textures in art paintings pose difficulties for sophisticated approaches like Generative Adversarial Networks (GANs), leading to issues such as small-scale texture synthesis and the inference of missing information, resulting in distortions in lines and unnatural colors. Simultaneously, patch-based image synthesis, augmented with global optimization on the image pyramid, has evolved to enhance structural coherence and details. However, gradient-based synthesis methods face obstacles related to directionality, inconsistency, and the computational burdens associated with solving the Poisson equation in non-integrable gradient fields. This paper introduces a pioneering approach, integrating Weighted Similarity-Confidence Laplacian Synthesis to comprehensively address these challenges and advance the field of artistic image completion. Experimental results affirm the effectiveness of our approach, offering promising outcomes for the preservation and restoration of art paintings with intricate details and irregular missing regions. The integration of weighted Laplacian synthesis and patch-based completion across multi-regions ensures precise and targeted completion, outperforming existing methods. A comparative analysis underscores our method’s superiority in artifact reduction and minimizing blurriness, particularly addressing challenges related to color discrepancies in texture areas. Additionally, the incorporation of pyramid blending proves advantageous, ensuring smoother transitions and preventing noticeable seams or artifacts in blended results. Based on empirical results, our method consistently outperforms previous methods across both high and low resolutions. Responding to these insights, our approach emerges as an invaluable guide for both curators and artists. The algorithm’s performance yields insights that underscore the central role of thoughtful decision making in the creation of art paintings. This guidance extends to informing choices related to color selection, brushstrokes, and various other elements integral to the artistic process. During the creation phase, employing these insights enables artists and curators to optimize not only the digitization but also the subsequent restoration process. This proves especially vital when dealing with the intricacies involved in physically restoring damaged original art paintings. Importantly, our approach not only streamlines the restoration process but also contributes significantly to the preservation and enhancement of the digital representations of these distinctive and often intricate works of art.

show abstract

“…Urano et al [39] attempted to upgrade structure propagation automatically by adopting auxiliary line construction containing penetrator and instructor lines separately to synthesize the intersection inside holes. Nurmala et al [6] and Horikawa et al [7] also struggled in automatic structure propagation by applying orthogonal viewpoints and clustered structure guidance of different perspective planes. Due to the implemented vanishing points of planar regions, they solved mainly straight lines, which did not support art painting based on brush strokes.…”

Section: Related Workmentioning

confidence: 99%

“…A large missing region is a challenging case in image completion. Many recent kinds of research attempted to solve this issue [6], [8], [9], [15], [37]. However, blurriness and color discrepancy appeared as burdensome.…”

Section: A Cohesive Laplacian Fusionmentioning

confidence: 99%

“…Pixel-based techniques [5] have produced impressive results in detail, but only for small missing regions such as blind spots or lines. Exemplar-based approaches, on the other hand, identify compatible and similar known patches in the surrounding hole area that can be replicated and applied to the receptive fields [6]- [14]. However, there is no guarantee that a accurate patch match will always exist in the known space, and errors such as color discrepancies may occur in the restored region if the appropriate patch is not found.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Structure-Texture Consistent Painting Completion for Artworks

Sari

2023

IEEE Access

View full text Add to dashboard Cite

Image completion techniques have made rapid and impressive progress due to advancements in deep learning and traditional patch-based approaches. The surrounding regions of a hole played a crucial role in repairing missing areas during the restoration process. However, large holes could result in suboptimal restoration outcomes due to complex textures causing significant changes in color gradations. As a result, they led to errors such as color discrepancies, blurriness, artifacts, and unnatural colors. Additionally, recent image completion approaches focused mainly on scenery and face images with fewer textures. Given these observations, we present a structure-texture consistent completion approach for filling large holes with detailed textures. Our method focuses on improving image completion in the context of artworks, which are expressions of creativity and often have more diverse structures and textures from applying paint to a surface using brush strokes. To handle the unique challenges posed by artwork, we segment non-homogeneous areas and then use Cohesive Laplacian Fusion to complete the texture of the remaining missing segmented area. This technique involves detecting changes in base structures and textures using multiple matched patches to achieve more consistent results. The experimental results show that our proposed method is competitive and outperforms state-of-the-art methods in missing regions and color gradations of art paintings.

show abstract

Image Inpainting using Orthogonal Viewpoints and Structure Consistency in Manhattan World

Cited by 7 publications

References 14 publications

Weighted Similarity-Confidence Laplacian Synthesis for High-Resolution Art Painting Completion

Weighted Similarity-Confidence Laplacian Synthesis for High-Resolution Art Painting Completion

Weighted Similarity-Confidence Laplacian Synthesis for High-Resolution Art Painting Completion

Structure-Texture Consistent Painting Completion for Artworks

Contact Info

Product

Resources

About