Visual Attribute Transfer through Deep Image Analogy

Liao, Jing; Yao, Yuan; Liu, Yuan; Hua, Gang; Kang, Sing Bing

doi:10.48550/arxiv.1705.01088

Cited by 62 publications

(76 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To alleviate these, optimization-based methods [2,45,4,68,44,37,36] typically formulate an objective function involving the data term and a prior term that favors similar correspondence fields among adjacent pixels with a balancing parameter λ, as shown in Fig. 2 (a), such that…”

Section: Motivationmentioning

confidence: 99%

“…Establishing dense correspondences across visually or semantically similar images facilitates a variety of computer vision applications [45,44,28,36]. Unlike sparse correspondence [48,5,77] that detects sparse points and finds matches across them, dense correspondence [45,6,34,36] aims at finding matches at each pixel and thus can benefit from prior 1 knowledge about matches among nearby pixels.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Matching Prior: Test-Time Optimization for Dense Correspondence

Hong¹,

Kim²

2021

2021 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Conventional techniques to establish dense correspondences across visually or semantically similar images focused on designing a task-specific matching prior, which is difficult to model. To overcome this, recent learningbased methods have attempted to learn a good matching prior within a model itself on large training data. The performance improvement was apparent, but the need for sufficient training data and intensive learning hinders their applicability. Moreover, using the fixed model at test time does not account for the fact that a pair of images may require their own prior, thus providing limited performance and poor generalization to unseen images.In this paper, we show that an image pair-specific prior can be captured by solely optimizing the untrained matching networks on an input pair of images. Tailored for such test-time optimization for dense correspondence, we present a residual matching network and a confidence-aware contrastive loss to guarantee a meaningful convergence. Experiments demonstrate that our framework, dubbed Deep Matching Prior (DMP), is competitive, or even outperforms, against the latest learning-based methods on several benchmarks for geometric matching and semantic matching, even though it requires neither large training data nor intensive learning. With the networks pre-trained, DMP attains state-of-the-art performance on all benchmarks.

show abstract

Section: Motivationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Deep Matching Prior: Test-Time Optimization for Dense Correspondence

Hong¹,

Kim²

2021

2021 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

show abstract

“…Leveraging the power of convolutional neural networks (CNNs) in learning highlevel semantic features, Long et al [43] first employ CNNs to establish semantic correspondences between images. Later efforts further improve correspondence quality by including additional annotations [6], [15], [16], [92], adopting coarse-to-fine strategy [37], extending to cross-domain images [87], etc. However, most existing studies only work with low-resolution correspondences as constrained by the heavy computation cost.…”

Section: Feature Correspondencementioning

confidence: 99%

Bi-level Feature Alignment for Versatile Image Translation and Manipulation

Zhan¹,

Yu²,

Wu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Generative adversarial networks (GANs) have achieved great success in image translation and manipulation. However, high-fidelity image generation with faithful style control remains a grand challenge in computer vision. This paper presents a versatile image translation and manipulation framework that achieves accurate semantic and style guidance in image generation by explicitly building a correspondence. To handle the quadratic complexity incurred by building the dense correspondences, we introduce a bi-level feature alignment strategy that adopts a top-k operation to rank block-wise features followed by dense attention between block features which reduces memory cost substantially. As the top-k operation involves index swapping which precludes the gradient propagation, we propose to approximate the non-differentiable top-k operation with a regularized earth mover's problem so that its gradient can be effectively back-propagated. In addition, we design a novel semantic position encoding mechanism that builds up coordinate for each individual semantic region to preserve texture structures while building correspondences. Further, we design a novel confidence feature injection module which mitigates mismatch problem by fusing features adaptively according to the reliability of built correspondences. Extensive experiments show that our method achieves superior performance qualitatively and quantitatively as compared with the state-of-the-art. The code is available at https://github.com/fnzhan/RABIT.

show abstract

“…The purpose of spatial correlation learning is to establish dense spatial correlation fields for image translation. Liao et al [16] proposed a coarse-to-fine strategy to compute the spatial correlation field for image analogy and style transfer. He et al [8] measured the spatial similarity between the reference and the target to perform exemplar-based colorization.…”

Section: Spatial Correlation Learningmentioning

confidence: 99%