Multitask Deep Learning With Spectral Knowledge for Hyperspectral Image Classification

Liu, Shengjie; Shi, Qian

doi:10.1109/lgrs.2019.2962768

Cited by 36 publications

(37 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The tail number here is not the global optimum and will be discussed in section IV-E. All results are reported from the average of ten random training sets. We compare our method with random forest, support vector machine, deep contextual CNN (DCCNN) [29], wide contextual residual network (WCRN) [73], the modified HResNet (the baseline of the proposed method) [43], and the state-of-the-art open-set method CROSR [52]. For open-set CNNs with confidence thresholding, the unknown probability is determined with a threshold of 0.5.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Few-Shot Hyperspectral Image Classification With Unknown Classes Using Multitask Deep Learning

Liu

Shi

Zhang

2021

IEEE Trans. Geosci. Remote Sensing

Self Cite

186

View full text Add to dashboard Cite

Current hyperspectral image classification assumes that a predefined classification system is closed and complete, and there are no unknown or novel classes in the unseen data. However, this assumption may be too strict for the real world. Often, novel classes are overlooked when the classification system is constructed. The closed nature forces a model to assign a label given a new sample and may lead to overestimation of known land covers (e.g., crop area). To tackle this issue, we propose a multitask deep learning method that simultaneously conducts classification and reconstruction in the open world (named MDL4OW) where unknown classes may exist. The reconstructed data are compared with the original data; those failing to be reconstructed are considered unknown, based on the assumption that they are not well represented in the latent features due to the lack of labels. A threshold needs to be defined to separate the unknown and known classes; we propose two strategies based on the extreme value theory for few-shot and manyshot scenarios. The proposed method was tested on real-world hyperspectral images; state-of-the-art results were achieved, e.g., improving the overall accuracy by 4.94% for the Salinas data. By considering the existence of unknown classes in the open world, our method achieved more accurate hyperspectral image classification, especially under the few-shot context.

show abstract

Section: Methodsmentioning

confidence: 99%

“…In addition, on the right side of the image, three groups of buildings (building-1, building-2, building-3) are not annotated in the original reference map. These buildings are often misclassified as natural land covers such as bare soil [35], [40]- [42] and meadows [43], or other unrelated materials such as asphalt [44], [45].…”

Section: Introductionmentioning

confidence: 99%

Few-Shot Hyperspectral Image Classification With Unknown Classes Using Multitask Deep Learning

Liu

Shi

Zhang

2021

IEEE Trans. Geosci. Remote Sensing

Self Cite

186

View full text Add to dashboard Cite

show abstract

“…CNN is widely used in HSI feature extraction tasks among these deep learning structures due to local perception, weight sharing, and other features. In literature [6], two-dimensional CNN is used as the basic module. Combined with multi-task learning strategies, two data sets are input for one model training, so that the network itself has more diverse feature recognition capabilities.…”

Section: Introductionmentioning

confidence: 99%

Dynamic Data Augmentation Method for Hyperspectral Image Classification Based on Siamese Structure

Zhang

Cao

Chen

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

At present, deep learning classification researches of hyperspectral usually focus on optimizing the classification model.In essence, most of them did not take special measures for the characteristics of the small sample and imbalanced category distribution of hyperspectral itself. Aiming at the problems of small samples and imbalanced category distribution, we propose a dynamic data selection algorithm. For one thing, this algorithm can dynamically select the samples that need data augmentation most. For another, it can be nested in Stochastic gradient descent (SGD) and can be easily implemented. Furthermore, there will be differences between the original sample and the transformed sample because of data augmentation transformation, which obstructs trained models' performance. Aiming at the difference between the augmented sample and the original sample, we define the similarity score and introduce the Siamese training structure to obtain the similarity score by which we reduce the difference through the SGD algorithm. Experiments show that the method proposed in this paper improves the classification results of the backbone training model when using data augmentation for training.

show abstract

“…In this study, we are going to use multitemporal SAR data from a whole year to produce crop maps, with both randomsampling training set and regional-sampling training set. We select three advanced CNNs, namely the wide contextual residual network (WCRN) [42], the HResNet [43], and the Double-Branch Multi-Attention Mechanism (DBMA) network [44], and random forest [45] to test their performance. Among the three deep learning models, the WCRN can run on CPU, and the DBMA has the best performance in benchmark datasets (b) Regional sampling.…”

Section: Introductionmentioning

confidence: 99%

Crop Mapping Using Sentinel Full-Year Dual-Polarized SAR Data and a CPU-Optimized Convolutional Neural Network With Two Sampling Strategies

Liu

Zhou

Ding

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

Self Cite

View full text Add to dashboard Cite

Although optical remote sensing can capture the Earth's environment with visible and infra-red sensors, it is limited by weather conditions. Often, only a few sets of cloudfree optical imagery are available in cloudy regions, where many agricultural towns are located. On the other hand, radar remote sensing can capture imagery under cloudy conditions. In this study, we examined the capability of Sentinel-1 multitemporal dual-polarized SAR imagery in a whole year from Google Earth Engine in crop mapping in two study sites in Chongqing, China, and Landivisiau, France. Results show that it is possible to produce better crop classification maps using multitemporal SAR imagery, but the performance is limited by local terrain. Flat agricultural regions, such as Western Europe, are expected to benefit from the multitemporal SAR information. Mountain agricultural regions, such as Southwestern China, will encounter difficulties due to the undulate terrain. We also tested two sampling strategies, i.e., random sampling and regional sampling, and observed high variation in overall accuracy: the former led to a higher accuracy. The gap is caused by the diversity of training sets examined using tSNE visualization. The importance of SAR channels in each month are correlated with their entropy. Data from the growing season are important in distinguishing crop types. 3D CNN achieved similar results under a huge computation cost compared with 2D CNNs. Based on the experiments, we recommend to use light-weight 2D CNN that can run on CPU for real-world crop mapping with SAR data.

show abstract

Multitask Deep Learning With Spectral Knowledge for Hyperspectral Image Classification

Cited by 36 publications

References 25 publications

Few-Shot Hyperspectral Image Classification With Unknown Classes Using Multitask Deep Learning

Few-Shot Hyperspectral Image Classification With Unknown Classes Using Multitask Deep Learning

Dynamic Data Augmentation Method for Hyperspectral Image Classification Based on Siamese Structure

Crop Mapping Using Sentinel Full-Year Dual-Polarized SAR Data and a CPU-Optimized Convolutional Neural Network With Two Sampling Strategies

Contact Info

Product

Resources

About