Progressive Reconstruction of Visual Structure for Image Inpainting

Li, Jinyuan; He, Fenghua; Zhang, Lefei; Du, Bo; Tao, Dacheng

doi:10.1109/iccv.2019.00606

Cited by 139 publications

(91 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Third, the learning process is also different, as illustrated in Section III-D part. We define the total reconstruction loss as the sum of the perceptual loss and pixel reconstruction loss [36], [63], [64] to generate visually indistinguishable images with input images, which is different from DRCN. By this means, we can ensure that the features of each layer in the encoder are selectively transmitted to the decoder and that the encoder can obtain transferable high-level features layer by layer.…”

Section: E Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Ladder-Suppression Network for Unsupervised Domain Adaptation

Deng

Zhao

Kuang

et al. 2022

IEEE Trans. Cybern.

View full text Add to dashboard Cite

Unsupervised domain adaptation (UDA) aims at learning a classifier for an unlabeled target domain by transferring knowledge from a labeled source domain with a related but different distribution. Most existing approaches learn domaininvariant features by adapting the entire information of the images. However, forcing adaptation of domain-specific variations undermines the effectiveness of the learned features. To address this problem, we propose a novel, yet elegant module, called the deep ladder-suppression network (DLSN), which is designed to better learn the cross-domain shared content by suppressing domain-specific variations. Our proposed DLSN is an autoencoder with lateral connections from the encoder to the decoder. By this design, the domain-specific details, which are only necessary for reconstructing the unlabeled target data, are directly fed to the decoder to complete the reconstruction task, relieving the pressure of learning domain-specific variations at the later layers of the shared encoder. As a result, DLSN allows the shared encoder to focus on learning cross-domain shared content and ignores the domain-specific variations. Notably, the proposed DLSN can be used as a standard module to be integrated with various existing UDA frameworks to further boost performance. Without whistles and bells, extensive experimental results on four gold-standard domain adaptation datasets, for example: 1) Digits; 2) Office31; 3) Office-Home; and 4) VisDA-C, demonstrate that the proposed DLSN can consistently and significantly improve the performance of various popular UDA frameworks.

show abstract

Section: E Discussionmentioning

confidence: 99%

“…The target-domain images are reconstructed using the ladder autoencoder. We define the total reconstruction loss L rec (θ en , θ de ) as the sum of the perceptual loss and pixel reconstruction loss [63], [64] to generate visually indistinguishable images with input images…”

Section: Network Trainingmentioning

confidence: 99%

Deep Ladder-Suppression Network for Unsupervised Domain Adaptation

Deng

Zhao

Kuang

et al. 2022

IEEE Trans. Cybern.

View full text Add to dashboard Cite

show abstract

“…However, the module may cause unwanted artifacts. Li et al [26] proposed Visual Structure Reconstruction (VSR), which can gradually add image structure information in image inpainting. However, it is not effective in the image of large irregular holes.…”

Section: B Learning-based Methodsmentioning

confidence: 99%

Image Inpainting With Learnable Edge-Attention Maps

et al. 2021

View full text Add to dashboard Cite

This paper proposes an end-to-end Learnable Edge-Attention Map (LEAM) method to assist image inpainting. To achieve a better-recovered effect, we design an edge attention module, which extracts the feature information of the edge map and re-normalizes the image feature information when automatically updating the edge map. And the information of known regions is adopted to assist the decoder generates semantically consistent results. A dual-discriminator structure consisting of the local discriminator and global discriminator is proposed to generate realistic texture details and improve the consistency of the overall structure. Experiments show that our method can obtain higher image inpainting quality than the existing state-of-the-art approaches, which improves PSNR by 3.58%, SSIM by 2.27%, and reduce MAE by 9.21% on average.INDEX TERMS Image inpainting, attention module, edge map, dual-discriminator.

show abstract

“…The learning-based methods, popular known as deep generative neural networks, have become the state of the art, based on their ability to learn distribution with regards to context. These approaches [9,10,11,12,13,14,15,16,17,18,19] use convolutional neural network (CNN) within an encoder-decoder within a GAN-based network to generate realistic images. These algorithms with a wide range of parameters and layers learn to manage feature extraction, propagation, and regularisation.…”

Section: Introductionmentioning

confidence: 99%

V-LinkNet: Learning Contextual Inpainting Across Latent Space of Generative Adversarial Network

Jam¹,

Kendrick²,

Drouard³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep learning methods outperform traditional methods in image inpainting. In order to generate contextual textures, researchers are still working to improve on existing methods and propose models that can extract, propagate, and reconstruct features similar to ground-truth regions. Furthermore, the lack of a high-quality feature transfer mechanism in deeper layers contributes to persistent aberrations on generated inpainted regions. To address these limitations, we propose the V-LinkNet cross-space learning strategy network. To improve learning on contextualised features, we design a loss model that employs both encoders. In addition, we propose a recursive residual transition layer (RSTL).The RSTL extracts high-level semantic information and propagates it down layers. Finally, we compare inpainting performance on the same face with different masks and on different faces with the same masks. To improve image inpainting reproducibility, we propose a standard protocol to overcome biases with various masks and images. We investigate the V-LinkNet components using experimental methods. Our result surpasses the state of the art when evaluated on the CelebA-HQ with the standard protocol. In addition, our model can generalise well when evaluated on Paris Street View, and Places2 datasets with the standard protocol.

show abstract

Progressive Reconstruction of Visual Structure for Image Inpainting

Cited by 139 publications

References 15 publications

Deep Ladder-Suppression Network for Unsupervised Domain Adaptation

Deep Ladder-Suppression Network for Unsupervised Domain Adaptation

Image Inpainting With Learnable Edge-Attention Maps

V-LinkNet: Learning Contextual Inpainting Across Latent Space of Generative Adversarial Network

Contact Info

Product

Resources

About