Difficulty-Aware Image Super Resolution via Deep Adaptive Dual-Network

Abstract

Recently, deep learning based single image superresolution(SR) approaches have achieved great development. The state-of-the-art SR methods usually adopt a feed-forward pipeline to establish a non-linear mapping between low-res(LR) and high-res(HR) images. However, due to treating all image regions equally without considering the difficulty diversity, these approaches meet an upper bound for optimization. To address this issue, we propose a novel SR approach that discriminately processes each image region within an image by its difficulty. Specifically, we propose a dual-way SR network that one way is trained to focus on easy image regions and another is trained to handle hard image regions. To identify whether a region is easy or hard, we propose a novel image difficulty recognition network based on PSNR prior. Our SR approach that uses the region mask to adaptively enforce the dual-way SR network yields superior results. Extensive experiments on several standard benchmarks (e.g., Set5, Set14, BSD100, and Urban100) show that our approach achieves state-of-the-art performance.

Depth Super-Resolution via Deep Controllable Slicing Network

“…To alleviate this problem, Zhang et al [42] proposed a lightweight ensemble method to improve the generalization of negative correlation learning for regression problems. Qin et al [31] suggested a difficulty-aware image SR method that use a dual-way network to separately recover easy image regions and hard ones. Li et al [23] applied a learning-based adapting method to ensemble the outputs from multiple models, which can exploit the information among successive video frames.…”

Wavelet-Based Dual-Branch Network for Image Demoireing

“…Dual-branch Design. Dual-branch network structure design makes use of two branches, complementing each other [8,40,6,45]. It has been used in image super-resolution [40], classification [8], segmentation [6] and person-re-ID [45].…”

Guided Dual Networks for Single Image Super-Resolution

“…DSRN [43] introduces a dual-state recurrent network to incorporate information from both the LR and the HR spaces. Dual-way SR [17] exploits a complex network EDSR as its complex branch and the bicubic interpolation as its plain branch to capture the global and the detail information. The above dual-path methods design different network structures for different branches to capture more information, but all the branches of these methods are trained with the same loss function.…”

“…The feature map is then processed by the sigmoid function to a probability matrix which enables the dual SR framework to yield superior results. Unlike other works that fuse the SR output of each branch to the final output [17], [46], we merge the feature maps in the process of SR image reconstruction which is before the final reconstruction convolution layer. The feature maps extracted from the mask network module is defined as:…”

“…In general, the existing dual branch SISR methods, such as DualCNN [16] and Dual-way SR [17], learn the global information and the details from two branches with different network structures. They are still PSNR-oriented methods and produce over-smoothed results as they use a single or the same loss function for both branches.…”