Hyperspectral and Multispectral Image Fusion Using Deep Convolutional Neural Network - ResNet Fusion

Abstract

In recent years, deep learning HS-MS fusion has become a very active research tool for the super resolution of hyperspectral image. The deep conventional neural networks (CNN) help to extract more detailed spectral and spatial features from the hyperspectral image. In CNN, each convolution layer takes the input from the previous layer which may cause the problems of information loss as the depth of the network increases. This loss of information causes vanishing gradient problems, particularly in the case of very high-resolution images. To overcome this problem in this work we propose a novel HS–MS ResNet fusion architecture with help of skip connection. The ResNet fusion architecture contains residual block with different stacked convolution layer, in this work we tested the residual block with two-, three-, and four- stacked convolution layers. To strengthens the gradients and for decreases negative effects from gradient vanishing, we implemented ResNet fusion architecture with different skip connections like short, long, and dense skip connection. We measure the strength and superiority of our ResNet fusion method against traditional methods by using four public datasets using standard quality measures and found that our method shows outstanding performance than all other compared methods.

Classification of wetland vegetation in the Yellow River Delta based on the fusion of hyperspectral data

“…The Yellow River Delta is situated within the bounds of Dongying City, Shandong Province, China (37°35'N～38°12'N, 118°33'E～119°20'E). It stretches northward to the Bohai Sea and is bordered to the south by Jiaozhou Bay, encompassing a total area of approximately 5400 km 2 . With an average elevation below 15m, the region experiences a temperate continental monsoon climate [1] .The eastern section of the Yellow River, i.e., the wetland area on both sides of the river channel at the mouth of the Yellow River, was selected for the study area (Figure 1).…”

“…[1] , and in recent years, in order to promote the high-quality development of the wetlands, new requirements have been put forward for the restoration of wetland vegetation resources. Remote sensing technology, as an emerging earth observation technology, can reflect the changes of things on the ground in real time and over a large area [2][3] . At present, the types of remote sensing data used to classify wetland vegetation mainly include multispectral images and hyperspectral images, but due to the trade-off between the design of the spatial and spectral resolution of the sensor, in which the multispectral images have medium spatial resolution and low spectral resolution, and the hyperspectral images have low spatial resolution and high spectral resolution, there is no sensor with both high spectral resolution and high spatial resolution at present [4][5] .…”