Explorable Super Resolution

Bahat, Yuval; Michaeli, Tomer

doi:10.1109/cvpr42600.2020.00279

Cited by 50 publications

(32 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is also inefficient that they have to train and store multiple separate models. On the other hand, Bahat et al [46] proposed an explorable SR framework that enables local restoration control. However, users have to manually edit the texture in a few steps through a user interface.…”

Section: Continuous Imagery Effect Transitionmentioning

confidence: 99%

Flexible Style Image Super-Resolution Using Conditional Objective

2022

View full text Add to dashboard Cite

Recent studies have significantly enhanced the performance of single-image super-resolution (SR) using convolutional neural networks (CNNs). While there can be many high-resolution (HR) solutions for a given input, most existing CNN-based methods do not explore alternative solutions during the inference. A typical approach to obtaining alternative SR results is to train multiple SR models with different loss weightings and exploit the combination of these models. Instead of using multiple models, we present a more efficient method to train a single adjustable SR model on various combinations of losses by taking advantage of multi-task learning. Specifically, we optimize an SR model with a conditional objective during training, where the objective is a weighted sum of multiple perceptual losses at different feature levels. The weights vary according to given conditions, and the set of weights is defined as a style controller. Also, we present an architecture appropriate for this training scheme, which is the Residual-in-Residual Dense Block equipped with spatial feature transformation layers. At the inference phase, our trained model can generate locally different outputs conditioned on the style control map. Extensive experiments show that the proposed SR model produces various desirable reconstructions without artifacts and yields comparable quantitative performance to state-of-the-art SR methods. Code and trained models are available at https://github.com/seungho-snu/FxSR.

show abstract

Section: Continuous Imagery Effect Transitionmentioning

confidence: 99%

Flexible Style Image Super-Resolution Using Conditional Objective

2022

View full text Add to dashboard Cite

show abstract

“…Randomness Under the settings of image restoration, many methods encourage diversity in their output by adding randomness [15,3,21]. In our setting, we may ask under what conditions there exists an optimal estimator X which is a deterministic function of Y .…”

Section: The Wasserstein and Gelbrich Distancesmentioning

confidence: 99%

“…However, it has recently been recognized that generating a single reconstructed image often does not convey to the user the inherent ambiguity in the problem. Therefore, many recent works target diverse perceptual image reconstruction, by employing randomization among possible restorations [15,3,21,1]. Commonly, such works perform sampling from the posterior distribution of natural images given the degraded input image.…”

Section: Introductionmentioning

confidence: 99%

A Theory of the Distortion-Perception Tradeoff in Wasserstein Space

Freirich¹,

Michaeli²,

Meir³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

The lower the distortion of an estimator, the more the distribution of its outputs generally deviates from the distribution of the signals it attempts to estimate. This phenomenon, known as the perception-distortion tradeoff, has captured significant attention in image restoration, where it implies that fidelity to ground truth images comes at the expense of perceptual quality (deviation from statistics of natural images). However, despite the increasing popularity of performing comparisons on the perception-distortion plane, there remains an important open question: what is the minimal distortion that can be achieved under a given perception constraint? In this paper, we derive a closed form expression for this distortion-perception (DP) function for the mean squared-error (MSE) distortion and the Wasserstein-2 perception index. We prove that the DP function is always quadratic, regardless of the underlying distribution. This stems from the fact that estimators on the DP curve form a geodesic in Wasserstein space. In the Gaussian setting, we further provide a closed form expression for such estimators. For general distributions, we show how these estimators can be constructed from the estimators at the two extremes of the tradeoff: The global MSE minimizer, and a minimizer of the MSE under a perfect perceptual quality constraint. The latter can be obtained as a stochastic transformation of the former.Preprint. Under review.

show abstract

“…The known importance of deeper CNNs in improving representational power has spurred the development of architectures that improve stability and provide better representations [18,24]. This is done not only through more residual connections [19,3,26], but also through structural preservation [27,16], constrained hypotheses spaces [7], fast Fourier transform [33] and generative techniques [35], and student-teacher networks [21]. Additionally, SR works that focus on improving contextual information have employed different attention mechanisms [29] and enhanced inception modules [30].…”

Section: Related Workmentioning

confidence: 99%

Thermal Image Super-Resolution Using Second-Order Channel Attention with Varying Receptive Fields

Gutierrez¹,

Beksi²

2021

Preprint

View full text Add to dashboard Cite

Thermal images model the long-infrared range of the electromagnetic spectrum and provide meaningful information even when there is no visible illumination. Yet, unlike imagery that represents radiation from the visible continuum, infrared images are inherently low-resolution due to hardware constraints. The restoration of thermal images is critical for applications that involve safety, search and rescue, and military operations. In this paper, we introduce a system to efficiently reconstruct thermal images. Specifically, we explore how to effectively attend to contrasting receptive fields (RFs) where increasing the RFs of a network can be computationally expensive. For this purpose, we introduce a deep attention to varying receptive fields network (AVRFN). We supply a gated convolutional layer with higher-order information extracted from disparate RFs, whereby an RF is parameterized by a dilation rate. In this way, the dilation rate can be tuned to use fewer parameters thus increasing the efficacy of AVRFN. Our experimental results show an improvement over the state of the art when compared against competing thermal image super-resolution (SR) methods.

show abstract

Explorable Super Resolution

Cited by 50 publications

References 17 publications

Flexible Style Image Super-Resolution Using Conditional Objective

Flexible Style Image Super-Resolution Using Conditional Objective

A Theory of the Distortion-Perception Tradeoff in Wasserstein Space

Thermal Image Super-Resolution Using Second-Order Channel Attention with Varying Receptive Fields

Contact Info

Product

Resources

About