Orientation-Aware Deep Neural Network for Real Image Super-Resolution

Abstract

Recently, Convolutional Neural Network (CNN) based approaches have achieved impressive single image superresolution (SISR) performance in terms of accuracy and visual effects. It is noted that most SISR methods assume that the low-resolution (LR) images are obtained through bicubic interpolation down-sampling, thus their performance on real-world LR images is limited. In this paper, we proposed a novel orientation-aware deep neural network (OA-DNN) model, which incorporate a number of orientation feature extraction and channel attention modules (OAMs), to achieve good SR performance on real-world LR images captured by a single-lens reflex (DSLR) camera. Orientation-aware features extracted in different directions are adaptively combined through a channel-wise attention mechanism to generate more distinctive features for high-fidelity recovery of image details. Moreover, we reshape the input image into smaller spatial size but deeper depth via an inverse pixel-shuffle operation to accelerate the training/testing speed without sacrificing restoration accuracy. Extensive experimental results indicate that our OA-DNN model achieves a good balance between accuracy and speed. The extended OA-DNN * + model further increases PSNR index by 0.18 dB compared with our previously submitted version. Codes will be made public after publication.

A Two-Stage Attentive Network for Single Image Super-Resolution

“…Attention in human perception generally means that human visual systems adaptively process visual information and focus on salient areas. Attention mechanisms have been widely applied in many tasks [48], [49], [50], [51], including image super-resolution [6], [26], [52], [21], [53]. Zhang et al [6] introduced attention mechanisms into the residual in residual structure to adaptively rescale channel-wise features for image super-resolution.…”

Super-Resolution using Deep Learning to Support Person Identification in Surveillance Video

“…In [13], the authors designed an SR system called Orientation-aware Deep Neural Network for Real Image Super-Resolution. The system uses as input real-world captured LR image after converting it to its corresponding Ychannel.…”

“…The authors in [14] introduced a system called Deep CNN with Skip Connection and Network in Network (DCSCN). Similarly, to [6] and [13], the network processes the original image's Y-channel and produces channels of corner pixels of each up-sampled pixel named 4ch (square of the scale factor channels). As displayed in Fig.…”

“…As reported above, various SR approaches based on deep learning have been proposed in the literature. Some of these approaches are designed for generic image SR [6,12,13,14,15] while others are designed for the specific application of faces images SR enhancement [16,17,18]. In the following, we discuss the reported approaches in terms of context, convergence, and scale factor.…”

“…Convergence: Since the LR image and the HR image are similar, learning the residual instead of learning the HR image reduces the required computations. The works reported in [8,13,14,15,16] learn residual images which speeds-up the training process. Scale-factor: for real-world applications, image scales differ naturally depending on the application.…”