Region Normalization for Image Inpainting

Yu, Tao; Guo, Zongyu; Jin, Xian-Min; Wu, Shilin; Chen, Zhibo; Li, Weiping; Zhang, Zhizheng; Liu, Sen

doi:10.1609/aaai.v34i07.6967

Cited by 155 publications

(105 citation statements)

References 22 publications

Supporting

Mentioning

105

Contrasting

Order By: Relevance

“…In the experiments, the irregular mask dataset in Reference [14] were used, and the PSNR and SSIM values were compared, as shown in Tables 5 and 6, where CA [15] represents the generative image inpainting results with Contextual Attention (CVPR2018), PC [14] is the results of Image Inpainting for Irregular Holes Using Partial Convolutions (ECCV2018), EC [17] represents the results of EdgeConnect (ICCV2019), GC [16] represents the results of Free-Form Image Inpainting with Gated Convolution (ICCV2019), LBAM [21] is the results of Learnable Bidirectional Attention Maps (ICCV2019), and RN [23] is the results of Region Normalization for Image Inpainting(AAAI2020). Among them, the data of PC comes from Reference [23,32], while others are performed using the codes or pre-trained models provided by their authors. We can see that the proposed network achieves much better metrics and surpasses the latest ones in terms of different ratios of mask area, meaning that our proposed method is more accurate.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Image Inpainting Using Two-Stage Loss Function and Global and Local Markovian Discriminators

Liu

et al. 2020

Sensors

View full text Add to dashboard Cite

Image inpainting networks can produce visually reasonable results in the damaged regions. However, existing inpainting networks may fail to reconstruct the proper structures or tend to generate the results with color discrepancy. To solve this issue, this paper proposes an image inpainting approach using the proposed two-stage loss function. The loss function consists of different Gaussian kernels, which are utilized in different stages of network. The use of our two-stage loss function in coarse network helps to focus on the image structure, while the use of it in refinement network is helpful to restore the image details. Moreover, we proposed a global and local PatchGANs (GAN means generative adversarial network), named GL-PatchGANs, in which the global and local markovian discriminators were used to control the final results. This is beneficial to focus on the regions of interest (ROI) on different scales and tends to produce more realistic structural and textural details. We trained our network on three popular datasets on image inpainting separately, both Peak Signal to Noise ratio (PSNR) and Structural Similarity (SSIM) between our results, and ground truths on test images show that our network can achieve better performance compared with the recent works in most cases. Besides, the visual results on three datasets also show that our network can produce visual plausible results compared with the recent works.

show abstract

Section: Methodsmentioning

confidence: 99%

“…In addition, Yang et al [22] considered the structure information in image generation network to produce the realistic structural images. Yu et al [23] proposed the region normalization for image inpainting and conducted the batch normalization in the damaged and undamaged areas, respectively.…”

Section: Image Inpainting Based On Deep Learningmentioning

confidence: 99%

Image Inpainting Using Two-Stage Loss Function and Global and Local Markovian Discriminators

Liu

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…All masks and images for training and testing are with the size of 256 × 256. We compare our method with five methods: PC [21], LBAM [23], GC [7], AN [19], RN [24]. Among them, GC adopts CA proposed in [6] at high-level feature maps for semantic inpainting.…”

Section: Experiments Settingsmentioning

confidence: 99%

Semantic-Aware Context Aggregation for Image Inpainting

Huang

Qin

Liu

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Recent attention-based image inpainting methods have made inspiring progress by propagating distant contextual information into holes. However, they tend to generate blurry contents since the propagation process is always misled by preliminarily-recovered holes features which are not well-inferred. To handle this problem, we propose a novel semantic-aware context aggregation module (SACA) that aggregates distant contextual information from a semantic perspective by exploiting the internal semantic similarity of the input feature map. Compared with existing attention mechanisms that model the relation of all pixel-pairs, SACA can suppress the impact of misleading holes features in context aggregation and significantly reduce computation burden by learning the relation between pixels and semantics. Also, we apply SACA to both high-level and low-level feature maps in our model for generating both semantically and visually plausible results. Extensive experiments on Outdoor Scenes, CelebA and Paris StreetView datasets validate the superiority of our method compared with existing methods.

show abstract

“…The authors claim that in their method, inference is possible irrespective of how the missing information is structured, while the state-of-the-art learningbased methods require specific information about the holes in the training phase. In [34], a spatial region-wise normalization named region normalization (RN) to overcome the limitation of image inpainting problem is proposed. The mean and variance shifts caused by full-spatial feature normalization (FN) limit the image inpainting network training is presented.…”

Section: Related Workmentioning

confidence: 99%

Region-Based Removal of Thermal Reflection Using Pruned Fully Convolutional Network

et al. 2020

View full text Add to dashboard Cite

In general, an image obtained from a thermal camera often has a mirror reflection or shadow reflected off the ground around an object, which is referred to as thermal reflection. Sometimes the thermal reflections are connected to their objects in images, which makes it difficult to detect or recognize the object only. Thermal reflections sometimes occur on the wall near an object and are detected as another object when they are not connected to the object. Furthermore, the size of thermal reflection and pixel value significantly vary with the medium of the reflected range and the surrounding temperature. In these cases, the patterns and pixel values of thermal reflection and the object become similar and difficult to distinguish. However, there are insufficient studies on removing the thermal reflection of various kinds of objects in diverse environments. Therefore, in this paper, we propose a pruned fully convolutional network (PFCN)-based method for removing the thermal reflection of an object using the surrounding information when image transformation is performed only within the region of an object. When experiments were conducted using self-collected databases (Dongguk thermal image database (DTh-DB) and Dongguk items & vehicles database (DI&V-DB)) and open databases, the method proposed herein exhibited more outstanding performance in removing thermal reflection when compared with the state-of-the-art methods. INDEX TERMS Thermal image, image transform, thermal reflection removal, pruned fully convolutional network.

show abstract

Region Normalization for Image Inpainting

Cited by 155 publications

References 22 publications

Image Inpainting Using Two-Stage Loss Function and Global and Local Markovian Discriminators

Image Inpainting Using Two-Stage Loss Function and Global and Local Markovian Discriminators

Semantic-Aware Context Aggregation for Image Inpainting

Region-Based Removal of Thermal Reflection Using Pruned Fully Convolutional Network

Contact Info

Product

Resources

About