Recurrently-Trained Super-Resolution

Abstract

We are motivated by the observation that for problems where inputs and outputs are in the same form such as in image enhancement, deep neural networks can be reinforced by retraining the network using a new target set to the output for the original target. As an example, we introduce a new learning strategy for super-resolution by recurrently training the same simple network. Unlike the existing self-trained SR, which involves a single stage of learning with multiple runs at test time, our method trains the same SR network multiple times with increasingly better targets requiring only a single inference at test time. At each stage of the proposed learning scheme, a new target for training is obtained by applying the most recently trained SR network to the original image and downscaling the resultant SR image to normalize the size. Even if downscaling is involved, we argue that the downscaled SR image acts as a better target compared to the old target. We could mathematically demonstrate that this process is similar to unsharp masking when it is linearly approximated and that this process makes the image sharper. However, unlike unsharp masking, the proposed recurrent learning tends to converge to a specific target. By retraining the existing network aiming at a more enhanced target, the proposed method can achieve a similar effect of applying SR multiple times without increasing implementation cost and inference time. To objectively verify the supremacy of our approach by experiments, we propose to use VIQET MOS, which does not require a reference image as a measure of image quality. As far as we know, our work of using an objective quality measure in image enhancement is the first one whose validity was verified by showing similar results to the actual user's subjective evaluation. The proposed recurrent learning scheme makes existing SR algorithms more useful by clearly improving the effect of SR.

A Review of Deep Learning Based Image Super-resolution Techniques

“…Mean square error (MSE) is a relatively traditional but widely used cost function. The mean square error is a cost function per pixel, and the mean square error is used as the cost function in RTSR [43]. RTSR believes that the quality of the high score image as the supervision image can be continuously improved through the trained super score network, so as to continuously supervise the training of the original super score network with the high score image with better quality, so as to improve the performance of the super score model.…”

Reverse image filtering with clean and noisy filters

“…In spite that the reverse image filtering idea was introduced very recently [30], it has already contributed to developing new deep learning architectures [6], studies on iterative depth B Alexander G. Belyaev a.belyaev@hw.ac.uk Lizhong Wang lw56@hw.ac.uk Pierre-Alain Fayolle fayolle@u-aizu.ac.jp 1 Heriot-Watt University, Edinburgh, UK 2 University of Aizu, Aizu-Wakamatsu, Japan and image super-resolution [22,35], and image restoration and sharpening [7,28].…”