Image Inpainting with Learnable Feature Imputation

Abstract

A regular convolution layer applying a filter in the same way over known and unknown areas causes visual artifacts in the inpainted image. Several studies address this issue with feature re-normalization on the output of the convolution. However, these models use a significant amount of learnable parameters for feature re-normalization [ 41 , 48 ], or assume a binary representation of the certainty of an output [ 11 , 26 ]. We propose (layer-wise) feature imputation of the missing input values to a convolution. In contrast to learned feature re-normalization [ 41 , 48 ], our method is efficient and introduces a minimal number of parameters. Furthermore, we propose a revised gradient penalty for image inpainting, and a novel GAN architecture trained exclusively on adversarial loss. Our quantitative evaluation on the FDF dataset reflects that our revised gradient penalty and alternative convolution improves generated image quality significantly. We present comparisons on CelebA-HQ and Places2 to current state-of-the-art to validate our model. (Code is available at: github.com/hukkelas/DeepPrivacy . Supplementary material can be downloaded from: folk.ntnu.no/haakohu/GCPR_supplementary.pdf ) Electronic supplementary material The online version of this chapter (10.1007/978-3-030-71278-5_28) contains supplementary material, which is available to authorized users.

Face Anonymity Based on Facial Pose Consistency

“…Ren et al (2018) trained a face anonymizer to delete private information and maximize the performance of spatial behavior detection. DeepPrivacy Hukkelås et al, (2020) generates authentic anonymous faces through existing background and sparse pose annotation. CIAGAN Maximov et al, (2020) uses facial key point coordinates and one hot coded identity vector to generate realistic anonymous faces while retaining other necessary features.…”

“…The method based on antagonistic disturbance (Kingma & Welling, 2013) (Sharif et al, 2016) usually highly depends on the reachability of the target system, requires special training, and has poor robustness. Recent generative-based methods (Hukkelås et al, 2020) (Chen et al, 2020;Guo & Chen, 2019;Hukkelås et al, 2019;Meden et al, 2017;Ren et al, 2018;Sun, Tewari, & Xu, 2018;Zhang, Hu, & Luo, 2018) also have difficulty generating realistic anonymous faces.…”