Information Leakage in Deep Learning-Based Hyperspectral Image Classification: A Survey

Feng, Hao; Wang, Yongcheng; Li, Zheng; Zhang, Ning; Zhang, Yuxi; Gao, Yunxiao

doi:10.3390/rs15153793

Cited by 6 publications

(5 citation statements)

References 94 publications

(125 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Feng [10] made a survey about information leakage in hyperspectral remote sensing image classification tasks. The work gathered some strategies to mitigate the effect of information leakage for spatial disjoint sampling, like a Cluster Sampling Strategy [11] that for each class, it is clustered into two clusters according to its spatial coordinates.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Analyzing Information Leakage on Video Object Detection Datasets by Splitting Images Into Clusters With High Spatiotemporal Correlation

Figueiredo,

Mendes

2024

IEEE Access

View full text Add to dashboard Cite

Random splitting strategy is a common approach for training, testing, and validating object detection algorithms based on deep learning. Is common for datasets to have images extracted from video sources, in which there are frames with high spatial correlation, i.e., frames with rotated positions or different view angles of the same object. These highly correlated frames may lead to information leakage in training, if these frames are not well-distributed. In this work, it is shown that datasets created with highly spatial correlation frames from the same video have information leakage if using the random splitting strategy to distribute the image into the sub-datasets. It proposed a clustering dataset split algorithm in which images are distributed randomly in the sub-datasets in a pack or clusters instead of a single image at the time. The clusters are created by extracting the image features from a video of the dataset using an image-text pre-trained model, CLIP, and reducing the feature vector dimensionality with t-Distributed Stochastic Neighbor embedding (t-SNE). In this reduced dimensional representation, images are separated into clusters using a clustering algorithms like DBSCAN, OPTICS, and Agglomerative Clustering. These clusters are distributed into the train, test, and validation datasets randomly to avoiding information leakage by highly spatial correlation frames. YOLOv8 is used as the object detector algorithm to test the dataset splitting.INDEX TERMS Data preprocessing, clustering, information leakage, supervised training, video annotation.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Most deep learning object detection algorithms use random sampling to train and test, but due to the temporal redundancy that frames from a video may have, some information about the test may be leaked. This affects the generalization ability of the model, making the model only learn the distribution of one domain [10].…”

Section: Introductionmentioning

confidence: 99%

Analyzing Information Leakage on Video Object Detection Datasets by Splitting Images Into Clusters With High Spatiotemporal Correlation

Figueiredo,

Mendes

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…In the urban context, the ability to classify the different elements of an urban landscape allows for efficient land-use planning. Urban sprawl can be monitored, and the encroachment of built-up areas on natural environments can be tracked with precision [8].…”

Section: Introductionmentioning

confidence: 99%

“…Multispectral (MS) and HS imaging are key techniques in agriculture for crop monitoring, disease detection, and yield estimation, capturing data at multiple wavelengths [3]. MS imaging uses fewer spectral bands (3)(4)(5)(6)(7)(8)(9)(10) covering broader wavelength ranges, making data processing easier and faster, suitable for real-time applications with limited resources. In contrast, HS imaging uses hundreds of narrower bands, providing finer spectral resolution and more detailed spectral signatures, but requires more complex data processing due to larger data volumes.…”

Section: Introductionmentioning

confidence: 99%

Enhancing Hyper-Spectral Image Classification with Reinforcement Learning and Advanced Multi-Objective Binary Grey Wolf Optimization

Shoeibi,

Nevisi,

Salehi

et al. 2024

CMC

View full text Add to dashboard Cite

Hyperspectral (HS) image classification plays a crucial role in numerous areas including remote sensing (RS), agriculture, and the monitoring of the environment. Optimal band selection in HS images is crucial for improving the efficiency and accuracy of image classification. This process involves selecting the most informative spectral bands, which leads to a reduction in data volume. Focusing on these key bands also enhances the accuracy of classification algorithms, as redundant or irrelevant bands, which can introduce noise and lower model performance, are excluded. In this paper, we propose an approach for HS image classification using deep Q learning (DQL) and a novel multi-objective binary grey wolf optimizer (MOBGWO). We investigate the MOBGWO for optimal band selection to further enhance the accuracy of HS image classification. In the suggested MOBGWO, a new sigmoid function is introduced as a transfer function to modify the wolves' position. The primary objective of this classification is to reduce the number of bands while maximizing classification accuracy. To evaluate the effectiveness of our approach, we conducted experiments on publicly available HS image datasets, including Pavia University, Washington Mall, and Indian Pines datasets. We compared the performance of our proposed method with several state-of-the-art deep learning (DL) and machine learning (ML) algorithms, including long short-term memory (LSTM), deep neural network (DNN), recurrent neural network (RNN), support vector machine (SVM), and random forest (RF). Our experimental results demonstrate that the Hybrid MOBGWO-DQL significantly improves classification accuracy compared to traditional optimization and DL techniques. MOBGWO-DQL shows greater accuracy in classifying most categories in both datasets used. For the Indian Pine dataset, the MOBGWO-DQL architecture achieved a kappa coefficient (KC) of 97.68% and an overall accuracy (OA) of 94.32%. This was accompanied by the lowest root mean square error (RMSE) of 0.94, indicating very precise predictions with minimal error. In the case of the Pavia University dataset, the MOBGWO-DQL model demonstrated outstanding performance with the highest KC of 98.72% and an impressive OA of 96.01%. It also recorded the lowest RMSE at 0.63, reinforcing its accuracy in predictions. The results clearly demonstrate that the proposed MOBGWO-DQL architecture not only reaches a highly accurate model more quickly but also maintains superior performance throughout the training process.

show abstract

“…This approach better captures diverse information from HSI data. Additionally, this article considers the issue of information leakage from overlap between the training set and the test set [36], as well as the challenge of achieving high classification accuracy for hyperspectral images with small sample sizes and imbalanced samples. This article introduces a double-branch multi-scale dualattention (DBMSDA) network for HSI classification.…”

mentioning

confidence: 99%

Hyperspectral Image Classification Based on Double-Branch Multi-Scale Dual-Attention Network

Zhang,

Liu,

Yang

et al. 2024

Remote Sensing

View full text Add to dashboard Cite

Although extensive research shows that CNNs achieve good classification results in HSI classification, they still struggle to effectively extract spectral sequence information from HSIs. Additionally, the high-dimensional features of HSIs, the limited number of labeled samples, and the common sample imbalance significantly restrict classification performance improvement. To address these issues, this article proposes a double-branch multi-scale dual-attention (DBMSDA) network that fully extracts spectral and spatial information from HSIs and fuses them for classification. The designed multi-scale spectral residual self-attention (MSeRA), as a fundamental component of dense connections, can fully extract high-dimensional and intricate spectral information from HSIs, even with limited labeled samples and imbalanced distributions. Additionally, this article adopts a dataset partitioning strategy to prevent information leakage. Finally, this article introduces a hyperspectral geological lithology dataset to evaluate the accuracy and applicability of deep learning methods in geology. Experimental results on the geological lithology hyperspectral dataset and three other public datasets demonstrate that the DBMSDA method exhibits superior classification performance and robust generalization ability compared to existing methods.

show abstract

Information Leakage in Deep Learning-Based Hyperspectral Image Classification: A Survey

Cited by 6 publications

References 94 publications

Analyzing Information Leakage on Video Object Detection Datasets by Splitting Images Into Clusters With High Spatiotemporal Correlation

Analyzing Information Leakage on Video Object Detection Datasets by Splitting Images Into Clusters With High Spatiotemporal Correlation

Enhancing Hyper-Spectral Image Classification with Reinforcement Learning and Advanced Multi-Objective Binary Grey Wolf Optimization

Hyperspectral Image Classification Based on Double-Branch Multi-Scale Dual-Attention Network

Contact Info

Product

Resources

About