Mutual encoder-decoder with bi-gated convolution for image inpainting

Abstract

Partial convolution and gated convolution (GC) have been widely used to solve the limitations of vanilla convolution. However, both approaches have their respective drawbacks. For example, the single-channel binary mask used in partial convolution restricts its flexibility, and the accuracy of the gating values learned from GC cannot be guaranteed. To overcome these limitations, we propose an approach called bigated convolution. It adaptively integrates the binary mask and the gating values learned from the network to obtain refined gating values that effectively characterize the features, thereby increasing the recognition accuracy of the gating value. Furthermore, we propose a feature-adaptive supplementation operation designed specifically for repairing damaged areas within the encoder features. Finally, we present a mutual encoder-decoder architecture with bi-gated convolution. Experiments conducted on two benchmark datasets show that the proposed method has the capability to generate visually plausible results.