Cross-Domain Correspondence Learning for Exemplar-Based Image Translation

Zhang, Pan; Zhang, Bo; Chen, Dong; Liu, Yuan; Wen, Fang

doi:10.1109/cvpr42600.2020.00519

Cited by 253 publications

(305 citation statements)

References 26 publications

Supporting

Mentioning

305

Contrasting

Order By: Relevance

“…Specially, it introduces a context preserving loss to learn identity function. CoCosNet [11] presents an end-to-end framework for exemplar-based image translation, and learns the dense semantic correspondence for cross-domain images by weakly supervised learning. A novel generative model named swapping autoencoder [82] shows excellent performance in image manipulation task, which encodes each image into two disentangled components and map the swapped features via an unsupervised manner.…”

Section: A Image-to-image Translationmentioning

confidence: 99%

“…In the first stage, the frames of target person F t and its extracted pose figures P t are fed into the generative adversarial network (GAN) to learn the mapping between the pose figures and person foreground. For the background texture, we use the spatially-adaptive denormalization (SPADE) block [83] to restore it by projecting the spatially structural style information to different activation locations [11]. In the second stage, we input the pose figures from source person P s and the frames of target person F t into the well-trained model to generate the person images I g under the source person's pose P s .…”

Section: Human Pose Transfer Network Architecture 1) Cross-domain mentioning

confidence: 99%

“…We convert the input images which from two different domains to a shared domain S where the both input semantics can be represented. Two domain adaptors without weight sharing are used to regulate the person images and the pose skeletons to a shared domain S. The domain adaptors comprise several Conv-InstanceNorm-LeakReLU blocks and the spatial size of features in S is 64 × 64 [11]. Concretely, the condition pose labels P t and the target person frames F t are sent to the feature pyramid network (FPN) to extract the features and transform them to the representations x S ∈ R W H×C and y S ∈ R W H×C in S This work is licensed under a Creative Commons Attribution 4.0 License.…”

Section: Human Pose Transfer Network Architecture 1) Cross-domain mentioning

confidence: 99%

“…For this reason, we match these intermediate representations from the pose images and person images according to M corr and obtain the coarse imageŷ I ∈ R W H×3 . As in [11], the blurred imageŷ I is acquired by selecting the most correlative pixels in y I and computing their weighted average.…”

Section: Human Pose Transfer Network Architecture 1) Cross-domain mentioning

confidence: 99%

“…Differentially from previous human motion transfer methods which almost use CycleGAN [7] or pix2pixHD [8] as global conditional generative adversarial networks (conditional GANs) [9], we present a new human pose transfer (HPT) [10] general framework based on cross-domain correspondence network (CoCosNet) [11] for pose-guided person image generation. Specifically, the generator comprises two sub-networks and learns cross-domain correspondence and image translation simultaneously by end-to-end learning.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Supervised Video-to-Video Synthesis for Single Human Pose Transfer

Wang

Huang

et al. 2021

IEEE Access

View full text Add to dashboard Cite

In this paper, we focus on human pose transfer in different videos, i.e., transferring the dance pose of a person in given video to a target person in the other video. Our methods can be summed up in three stages to tackle this challenging scenario. Firstly, we extract the frames and pose masks from the source video and target video. Secondly, we use our model to synthesize the frames of target person with the given dance pose. Thirdly, we refine the generated frames to improve the quality of outputs. Our model is built on three stages: 1) human pose extraction and normalization. 2) a GAN based on cross-domain correspondence mechanism to synthesize dance-guided person image in target video by consecutive frames and pose stick images. 3) coarse-to-fine generation strategy which includes two GANs: a GAN used to reconstruct human face in target video, the other generates smoothing frame sequences. Finally, we compress the sequential frames generated from our model into video format. Compared with previous works, our model manifests better person appearance consistency and time coherence in video-to-video synthesis for human motion transfer, which makes the generated video look more realistic. The qualitative and quantitative comparisons represent our approach performs significant improvements over the state-of-the-art methods. Experiments on synthetic frames and ground truth validate the effectiveness of the proposed method. INDEX TERMS Generative adversarial network (GAN), image-to-image translation, video-to-video synthesis, pose-guided person image generation

show abstract

Section: A Image-to-image Translationmentioning

confidence: 99%

Section: Human Pose Transfer Network Architecture 1) Cross-domain mentioning

confidence: 99%

Section: Human Pose Transfer Network Architecture 1) Cross-domain mentioning

confidence: 99%

Section: Human Pose Transfer Network Architecture 1) Cross-domain mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Supervised Video-to-Video Synthesis for Single Human Pose Transfer

Wang

Huang

et al. 2021

IEEE Access

View full text Add to dashboard Cite

show abstract

Controlling StyleGANs using rough scribbles via one‐shot learning

Endo

Kanamori

2022

Computer Animation & Virtual

View full text Add to dashboard Cite

This paper tackles the challenging problem of one-shot semantic image synthesis from rough sparse annotations, which we call "semantic scribbles." Namely, from only a single training pair annotated with semantic scribbles, we generate realistic and diverse images with layout control over, for example, facial part layouts and body poses. We present a training strategy that performs pseudo labeling for semantic scribbles using the StyleGAN prior. Our key idea is to construct a simple mapping between StyleGAN features and each semantic class from a single example of semantic scribbles. With such mappings, we can generate an unlimited number of pseudo semantic scribbles from random noise to train an encoder for controlling a pretrained StyleGAN generator. Even with our rough pseudo semantic scribbles obtained via one-shot supervision, our method can synthesize high-quality images thanks to our GAN inversion framework.We further offer optimization-based postprocessing to refine the pixel alignment of synthesized images. Qualitative and quantitative results on various datasets demonstrate improvement over previous approaches in one-shot settings.

show abstract

Personalized facial makeup transfer based on outline correspondence

Wang

2023

Computer Animation & Virtual

View full text Add to dashboard Cite

Most existing makeup transfer techniques focus on light makeup styles, which limits the task of makeup transfer to color manipulation issues such as eye shadow and lip gloss. However, the makeup in real life is diverse and personalized, not only the most basic foundation, eye makeup, but also the painted patterns on the face, jewelry decoration and other personalized makeup. Inspired by the painting steps of drawing the outline first and then coloring, we propose a makeup transfer network for personalized makeup, which realizes face makeup transfer by learning outline correspondence. Specifically, we propose the outline feature extraction module and outline loss that can promote outline correspondence. Our network can not only transfer daily light makeup, but also complete transfer for complex facial painting patterns. Experiments show that our method can obtain visually more accurate makeup transfer results. Quantitative and qualitative experimental results show that the method proposed in this paper achieves superior results in extreme makeup transfer compared to the state‐of‐the‐art methods.

show abstract

Cross-Domain Correspondence Learning for Exemplar-Based Image Translation

Cited by 253 publications

References 26 publications

Supervised Video-to-Video Synthesis for Single Human Pose Transfer

Supervised Video-to-Video Synthesis for Single Human Pose Transfer

Controlling StyleGANs using rough scribbles via one‐shot learning

Personalized facial makeup transfer based on outline correspondence

Contact Info

Product

Resources

About