Guided Image-to-Image Translation with Bi-Directional Feature Transformation

Abstract

Pose Transfer Texture Transfer Upsampling Input Guide Ours Input Guide Ours Figure 1. Applications of guided image-to-image translation. We present an algorithm that translates an input image into a corresponding output image while respecting the constraints specified in the provided guidance image. These controllable image-to-image translation problems often require task-specific architectures and training objective functions as the guidance can take various different forms (e.g., color strokes, sketch, texture patch, image, and mask). We introduce a new conditioning scheme for controlling image synthesis using available guidance signals and demonstrate applicability to several sample applications, including person image synthesis guided by a given pose (top), sketch-to-photo synthesis guided with a texture patch (middle), and depth upsampling guided with an RGB image (bottom).