Unsupervised Real Image Super-Resolution via Generative Variational AutoEncoder

Liu, Zhi Song; Siu, Wan Chi; Wang, Li Wen; Li, Chu Tak; Cani, Marie Paule; Chan, Yui-Lam

doi:10.1109/cvprw50498.2020.00229

Cited by 32 publications

(23 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To better evaluate the visual quality of view rendering than just visualizing the generated images, we used LPIPS [54] to measure the deep feature similarity. It is widely used in image processing tasks [55], [56], [57]. Using HRNet [53], we also estimated the semantic segmentation map from the generated image to compare with the ground truth by averaging pixel accuracy (%).…”

Section: Evaluation Metricsmentioning

confidence: 99%

See360: Novel Panoramic View Interpolation

Liu

Cani

Siu

2022

IEEE Trans. on Image Process.

Self Cite

View full text Add to dashboard Cite

We present See360, which is a versatile and efficient framework for 360 • panoramic view interpolation using latent space viewpoint estimation. Most of the existing view rendering approaches only focus on indoor or synthetic 3D environments and render new views of small objects. In contrast, we suggest to tackle camera-centered view synthesis as a 2D affine transformation without using point clouds or depth maps, which enables an effective 360 • panoramic scene exploration. Given a pair of reference images, the See360 model learns to render novel views by a proposed novel Multi-Scale Affine Transformer (MSAT), enabling the coarse-to-fine feature rendering. We also propose a Conditional Latent space AutoEncoder (C-LAE) to achieve view interpolation at any arbitrary angle. To show the versatility of our method, we introduce four training datasets, namely Ur-banCity360, Archinterior360, HungHom360 and Lab360, which are collected from indoor and outdoor environments for both real and synthetic rendering. Experimental results show that the proposed method is generic enough to achieve real-time rendering of arbitrary views for all four datasets. In addition, our See360 model can be applied to view synthesis in the wild: with only a short extra training time (approximately 10 mins), and is able to render unknown real-world scenes. The superior performance of See360 opens up a promising direction for camera-centered view rendering and 360 • panoramic view interpolation.

show abstract

Section: Evaluation Metricsmentioning

confidence: 99%

See360: Novel Panoramic View Interpolation

Liu

Cani

Siu

2022

IEEE Trans. on Image Process.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Instead, we embed the variation module into the transformation model to map features into a linear space spanned by mixture models. VAE is defined as: P (X)= P (X, z)P (z|X) dz, where X is the input and z is sampled from the latent space Z [24,26,27]. To regularize the latent space, we use the Kullback-Leibler (KL) divergence that measures the probability close to a normal distribution.…”

Section: Variation Modulementioning

confidence: 99%

Multiple Style Transfer Via Variational Autoencoder

Liu

Kalogeiton

Cani

2021

2021 IEEE International Conference on Image Processing (ICIP)

Self Cite

View full text Add to dashboard Cite

Modern works on style transfer focus on transferring style from a single image. Recently, some approaches study multiple style transfer; these, however, are either too slow or fail to mix multiple styles. We propose ST-VAE, a Variational Au-toEncoder for latent space-based style transfer. It performs multiple style transfer by projecting nonlinear styles to a linear latent space, enabling to merge styles via linear interpolation before transferring the new style to the content image. To evaluate ST-VAE, we experiment on COCO for single and multiple style transfer. We also present a case study revealing that ST-VAE outperforms other methods while being faster, flexible, and setting a new path for multiple style transfer.

show abstract

“…4). Recently, there are some new attempts for blind SR. Several CycleGAN [47] based methods [4,40,21,20] learn from unpaired LR-HR images, but they are more difficult to train. ZSSR [29] explores the zero-shot solution for the first time, where the CNN learns the mapping from the LR image and its downscaled versions (self-supervision).…”

Section: Introductionmentioning

confidence: 99%

Conditional Hyper-Network for Blind Super-Resolution with Multiple Degradations

Yin,

Wang,

Yuan

et al. 2021

Preprint

View full text Add to dashboard Cite

Although single-image super-resolution (SISR) methods have achieved great success on single degradation, they still suffer performance drop with multiple degrading effects in real scenarios. Recently, some blind and non-blind models for multiple degradations have been explored. However, those methods usually degrade significantly for distribution shifts between the training and test data. Towards this end, we propose a conditional meta-network framework (named CMDSR) for the first time, which helps SR framework learn how to adapt to changes in input distribution. We extract degradation prior at task-level with the proposed Condi-tionNet, which will be used to adapt the parameters of the basic SR network (BaseNet). Specifically, the ConditionNet of our framework first learns the degradation prior from a support set, which is composed of a series of degraded image patches from the same task. Then the adaptive BaseNet rapidly shifts its parameters according to the conditional features. Moreover, in order to better extract degradation prior, we propose a task contrastive loss to decrease the inner-task distance and increase the cross-task distance between task-level features. Without predefining degradation maps, our blind framework can conduct one single parameter update to yield considerable SR results. Extensive experiments demonstrate the effectiveness of CMDSR over various blind, even non-blind methods. The flexible BaseNet structure also reveals that CMDSR can be a general framework for large series of SISR models.

show abstract

Unsupervised Real Image Super-Resolution via Generative Variational AutoEncoder

Cited by 32 publications

References 32 publications

See360: Novel Panoramic View Interpolation

See360: Novel Panoramic View Interpolation

Multiple Style Transfer Via Variational Autoencoder

Conditional Hyper-Network for Blind Super-Resolution with Multiple Degradations

Contact Info

Product

Resources

About