Accelerating the Super-Resolution Convolutional Neural Network

Dong, Chao; Loy, Chen Change; Tang, Xiaoou

doi:10.1007/978-3-319-46475-6_25

Cited by 2,488 publications

(2,083 citation statements)

References 26 publications

Supporting

Mentioning

1,873

Contrasting

Unclassified

Order By: Relevance

“…CNN has been successfully applied to spatial enhancement of single images [29][30][31][32][33]. In [29], Dong et al proposed a super-resolution CNN network (SRCNN).…”

Section: Background Of Cnn Based Image Super-resolutionmentioning

confidence: 99%

“…CNN has Inspired by this idea, some other CNN based super-resolution methods have also been proposed. For example, a faster SRCNN (FSRCNN) was proposed by adopting a deconvolution layer and small kernel size in [30]. Kim et al [31] pointed out that increasing the depth of CNN is helpful for improving the super-resolution performance.…”

Section: Background Of Cnn Based Image Super-resolutionmentioning

confidence: 99%

“…The purpose of up-scaling is to match the size of LR HSI to that of HR MSI and HR HSI. This strategy has also been adopted in other deep learning-based image super-resolution methods, for example, in [29,30,34]. The deep learning model is trained to learn the mapping between up-scaled HSI and HR It is noted that F HSI l (s LR n ) and F MSI l (P HR n ) are obtained by vectorizing the feature maps of the HSI and MSI branches respectively.…”

Section: Architecture Of the Two Branches Cnn For Fusionmentioning

confidence: 99%

See 2 more Smart Citations

Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network

2018

View full text Add to dashboard Cite

Abstract:Enhancing the spatial resolution of hyperspectral image (HSI) is of significance for applications. Fusing HSI with a high resolution (HR) multispectral image (MSI) is an important technology for HSI enhancement. Inspired by the success of deep learning in image enhancement, in this paper, we propose a HSI-MSI fusion method by designing a deep convolutional neural network (CNN) with two branches which are devoted to features of HSI and MSI. In order to exploit spectral correlation and fuse the MSI, we extract the features from the spectrum of each pixel in low resolution HSI, and its corresponding spatial neighborhood in MSI, with the two CNN branches. The extracted features are then concatenated and fed to fully connected (FC) layers, where the information of HSI and MSI could be fully fused. The output of the FC layers is the spectrum of the expected HR HSI. In the experiment, we evaluate the proposed method on Airborne Visible Infrared Imaging Spectrometer (AVIRIS), and Environmental Mapping and Analysis Program (EnMAP) data. We also apply it to real Hyperion-Sentinel data fusion. The results on the simulated and the real data demonstrate that the proposed method is competitive with other state-of-the-art fusion methods.

show abstract

“…CNN has been successfully applied to spatial enhancement of single images [29][30][31][32][33]. In [29], Dong et al proposed a super-resolution CNN network (SRCNN).…”

Section: Background Of Cnn Based Image Super-resolutionmentioning

confidence: 99%

Section: Background Of Cnn Based Image Super-resolutionmentioning

confidence: 99%

Section: Architecture Of the Two Branches Cnn For Fusionmentioning

confidence: 99%

See 1 more Smart Citation

Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network

2018

View full text Add to dashboard Cite

show abstract

“…During 1980s, (Huang & Tsay 1984) enhanced the resolution of scenes received from Landsat TM satellite. Until recently, (Romano et al 2016) , (Dong et al 2016) and (Dong et al 2015) the trend of SR evolved totally towards the example-based methods. Example-based methods use a dictionary of mapping between LR and HR to infer the unknown HR details (Bevilacqua et al 2012) It exploits the self-similarity and generate patches from the input images.…”

Section: Super-resolutionmentioning

confidence: 99%

Multi-Modal Remote Sensing Data Fusion Framework

Ghaffar¹,

Vu²,

Maul³

2017

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

ABSTRACT:The inconsistency between the freely available remote sensing datasets and crowd-sourced data from the resolution perspective forms a big challenge in the context of data fusion. In classical classification problems, crowd-sourced data are represented as points that may or not be located within the same pixel. This discrepancy can result in having mixed pixels that could be unjustly classified. Moreover, it leads to failure in retaining sufficient level of details from data inferences. In this paper we propose a method that can preserve detailed inferences from remote sensing datasets accompanied with crowd-sourced data. We show that advanced machine learning techniques can be utilized towards this objective. The proposed method relies on two steps, firstly we enhance the spatial resolution of the satellite image using Convolutional Neural Networks and secondly we fuse the crowd-sourced data with the upscaled version of the satellite image. However, the covered scope in this paper is concerning the first step. Results show that CNN can enhance Landsat 8 scenes resolution visually and quantitatively.

show abstract

“…Recently, they further proposed an improved version of SRCNN that takes the 1 × 1 convolution into account to reduce the network weights and result in a fast SRCNN (FSRCNN) [12]. Different from [9][10][11][12] that use the undegraded image as ground true for training, some works try to learn image residual. Kim et al [17] proposed a very deep network to learn residual to accelerate the convergence speed.…”

Section: Introductionmentioning

confidence: 99%

Anchored neighborhood deep network for single-image super-resolution

Shi

Liu

Jiang

et al. 2018

J Image Video Proc.

View full text Add to dashboard Cite

Real-time image and video processing is a challenging problem in smart surveillance applications. It is necessary to trade off between high frame rate and high resolution to meet the limited bandwidth requirement in many specific applications. Thus, image super-resolution become one commonly used techniques in surveillance platform. The existing image super-resolution methods have demonstrated that making full use of image prior can improve the algorithm performance. However, the previous deep-learning-based image super-resolution methods rarely take image prior into account. Therefore, how to make full use of image prior is one of the unsolved problems for deep-network-based single image super-resolution methods. In this paper, we establish the relationship between the traditional sparse-representation-based single-image super-resolution methods and the deep-learning-based ones and use transfer learning to make our proposed deep network take the image prior into account. Another unresolved problem of the deep-learning-based single-image super-resolution method is how to avoid neurons compromise to different image contents. In this paper, the image patches are anchored to the dictionary atoms to group into various categories. As a result, each neuron will work on the same types of image patches that have similar details, which makes the network more accurate to recover high-frequency details. By solving these two problems, we propose an anchored neighborhood deep network for single-image super-resolution. Experimental results show that our proposed method outperforms many state-of-the-art single-image super-resolution methods.

show abstract

Accelerating the Super-Resolution Convolutional Neural Network

Cited by 2,488 publications

References 26 publications

Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network

Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network

Multi-Modal Remote Sensing Data Fusion Framework

Anchored neighborhood deep network for single-image super-resolution

Contact Info

Product

Resources

About