Explorable Super Resolution

Bahat, Yuval; Michaeli, Tomer

doi:10.48550/arxiv.1912.01839

Cited by 3 publications

(10 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Explorable image restoration The idea of explorable image restoration was presented in [3] and further studied in [4], but had not been applied to domains other than super-resolution to date. In the context of image compression, Guo and Chao [16] presented a method that can generate diverse outputs.…”

Section: Related Workmentioning

confidence: 99%

“…We define our control signal z ∈ R m×n×64 to have the same dimensions as x Q , so as to allow intricate editing abilities. Following the practice in [3], we concatenate z to the input of each of the N layers of our network, to promote faster training.…”

Section: Our Decoder Designmentioning

confidence: 99%

“…We train our model following the general procedure of [3]. As an initialization step, we train it to minimize the L 1 distance between ground truth, uncompressed training images, and the corresponding outputs of our network, while randomly drawing the QF parameter for each image.…”

Section: Our Decoder Designmentioning

confidence: 99%

“…We train our model on 1.15M images from the ImageNet training set [29], using batches of 16 with an Adam optimizer with learning rates of 0.0001 and 0.00001 for the initialization and consecutive training phases, respectively. To create training input codes, we compress the GT training images utilizing a quantization interval matrix M = QF • Q baseline /5000, where QF is independently sampled from a uniform distribution over [5,49] for each image 3 , and Q baseline is the example baseline table in the JPEG standard [33]. We use N = 10 layers for both the generator and the critic models, using 320 output channels for all convolution operations but the last.…”

Section: Training Detailsmentioning

confidence: 99%

“…for image synthesis [6,19,38] and image reconstruction [16] tasks. In the context of super resolution, the works of [3,4] took another step further, and allowed a user to traverse the space of high resolution images that correspond to a given low resolution input. In this paper, we propose to adapt this approach to the realm of visual decompression, and specifically illustrate explorable image decompression for the ubiquitous JPEG standard.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

What’s in the Image? Explorable Decoding of Compressed Images

Bahat

Michaeli

2021

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

View full text Add to dashboard Cite

The ever-growing amounts of visual contents captured on a daily basis necessitate the use of lossy compression methods in order to save storage space and transmission bandwidth. While extensive research efforts are devoted to improving compression techniques, every method inevitably discards information. Especially at low bit rates, this information often corresponds to semantically meaningful visual cues, so that decompression involves significant ambiguity. In spite of this fact, existing decompression algorithms typically produce only a single output, and do not allow the viewer to explore the set of images that map to the given compressed code. Recently, explorable image restoration has been studied in the context of super-resolution. In this work, we propose to take this idea to the realm of image decompression. Specifically, we develop a novel deep-network based decoder architecture for the ubiquitous JPEG standard, which allows traversing the set of decompressed images that are consistent with the compressed input code. To allow for simple user interaction, we also develop a graphical user interface that comprises several intuitive exploration and editing tools. We exemplify our framework on graphical, medical and forensic use cases, demonstrating its wide range of potential applications.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Our Decoder Designmentioning

confidence: 99%

Section: Our Decoder Designmentioning

confidence: 99%

Section: Training Detailsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

What’s in the Image? Explorable Decoding of Compressed Images

Bahat

Michaeli

2021

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

View full text Add to dashboard Cite

show abstract

Sparsity Aware Normalization for GANs

Kligvasser

Michaeli

2021

AAAI

Self Cite

View full text Add to dashboard Cite

Generative adversarial networks (GANs) are known to benefit from regularization or normalization of their critic (discriminator) network during training. In this paper, we analyze the popular spectral normalization scheme, find a significant drawback and introduce sparsity aware normalization (SAN), a new alternative approach for stabilizing GAN training. As opposed to other normalization methods, our approach explicitly accounts for the sparse nature of the feature maps in convolutional networks with ReLU activations. We illustrate the effectiveness of our method through extensive experiments with a variety of network architectures. As we show, sparsity is particularly dominant in critics used for image-to-image translation settings. In these cases our approach improves upon existing methods, in less training epochs and with smaller capacity networks, while requiring practically no computational overhead.

show abstract