Rapid, detail-preserving image downscaling

Weber, Nicolas; Waechter, Michael; Amend, Sandra C.; Guthe, Stefan; Goesele, Michael

doi:10.1145/2980179.2980239

Cited by 59 publications

(64 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This section reports the quantitative and qualitative performance of different image downscaling methods for SR. Then ablation studies of the proposed CAR model is conducted. We compare the CAR model with four baseline methods, i.e., the bicubic downscaling (Bicubic), and other three state-of-the-art image downscaling methods: perceptually optimized image downscaling (Perceptually) [15], detail-preserving image downscaling (DPID) [17], and L0-regularized image downscaling (L0-regularized) [19]. We train SR models using LR images downscaled by those four baseline downscaling algorithms and LR images downscaled by the proposed CAR model.…”

Section: Evaluation Of Downscaling Methods For Srmentioning

confidence: 99%

“…We adopt similar evaluation settings used in [14,15,17,19]. The user study are conducted as the A/B testing: the original image is presented in the middle place with two variants (SR images or downscaled images) showed in either side, among which one is produced by the CAR method and another is generated by one of the competing methods, i.e., Bicubic, Perceptually [15], DPID [17] and L0-regularized [19]. The users are required to ask the question 'which one looks better' by exclusively selecting one of the three options from: 1) A is better than B; 2) A equals to B; 3) B is better than A.…”

Section: User Studymentioning

confidence: 99%

See 1 more Smart Citation

Learned Image Downscaling for Upscaling Using Content Adaptive Resampler

Sun

Chen

2020

IEEE Trans. on Image Process.

133

View full text Add to dashboard Cite

Deep convolutional neural network based image super-resolution (SR) models have shown superior performance in recovering the underlying high resolution (HR) images from low resolution (LR) images obtained from the predefined downscaling methods. In this paper we propose a learned image downscaling method based on content adaptive resampler (CAR) with consideration on the upscaling process. The proposed resampler network generates content adaptive image resampling kernels that are applied to the original HR input to generate pixels on the downscaled image. Moreover, a differentiable upscaling (SR) module is employed to upscale the LR result into its underlying HR counterpart. By back-propagating the reconstruction error down to the original HR input across the entire framework to adjust model parameters, the proposed framework achieves a new state-of-the-art SR performance through upscaling guided image resamplers which adaptively preserve detailed information that is essential to the upscaling. Experimental results indicate that the quality of the generated LR image is comparable to that of the traditional interpolation based method, but the significant SR performance gain is achieved by deep SR models trained jointly with the CAR model. The code is publicly available on: URL https://github.com/sunwj/CAR.

show abstract

Section: Evaluation Of Downscaling Methods For Srmentioning

confidence: 99%

Section: User Studymentioning

confidence: 99%

Learned Image Downscaling for Upscaling Using Content Adaptive Resampler

Sun

Chen

2020

IEEE Trans. on Image Process.

133

View full text Add to dashboard Cite

show abstract

“…Here, we focus on downscaling inside the network and propose a pooling layer that is trainable, fully differentiable, and includes major pooling techniques as special (limit) Figure 2. Diagram of detail-preserving downscaling (DPID) [31] and our detail-preserving pooling (DPP). DPP omits the Gaussian filter; Full-DPP replaces the box filter with a learned 2D filter.…”

Section: Related Workmentioning

confidence: 99%

“…cases. Moreover, we build on a detail-preserving image downscaling approach [31] that outperforms conventional image downscaling techniques in subjective testing.…”

Section: Related Workmentioning

confidence: 99%

“…In contrast to traditional downscaling algorithms that aim for physically plausible results, more recent work has shown benefits of focusing on aspects of human perception. We specifically consider the approach of Weber et al [31], which aims to preserve small details of the input image, which are often crucial for a faithful visual impression ( Fig. 1).…”

Section: Detail-preserving Image Downscalingmentioning

confidence: 99%

See 1 more Smart Citation

Detail-Preserving Pooling in Deep Networks

Saeedan

Weber

Goesele

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

Self Cite

112

View full text Add to dashboard Cite

Most convolutional neural networks use some method for gradually downscaling the size of the hidden layers. This is commonly referred to as pooling, and is applied to reduce the number of parameters, improve invariance to certain distortions, and increase the receptive field size. Since pooling by nature is a lossy process, it is crucial that each such layer maintains the portion of the activations that is most important for the network's discriminability. Yet, simple maximization or averaging over blocks, max or average pooling, or plain downsampling in the form of strided convolutions are the standard. In this paper, we aim to leverage recent results on image downscaling for the purposes of deep learning. Inspired by the human visual system, which focuses on local spatial changes, we propose detailpreserving pooling (DPP), an adaptive pooling method that magnifies spatial changes and preserves important structural detail. Importantly, its parameters can be learned jointly with the rest of the network. We analyze some of its theoretical properties and show its empirical benefits on several datasets and networks, where DPP consistently outperforms previous pooling approaches.

show abstract