Deep Learning Methods for Semantic Segmentation in Remote Sensing with Small Data: A Survey

Abstract

The annotations used during the training process are crucial for the inference results of remote sensing images (RSIs) based on a deep learning framework. Unlabeled RSIs can be obtained relatively easily. However, pixel-level annotation is a process that necessitates a high level of expertise and experience. Consequently, the use of small sample training methods has attracted widespread attention as they help alleviate reliance on large amounts of high-quality labeled data and current deep learning methods. Moreover, research on small sample learning is still in its infancy owing to the unique challenges faced when completing semantic segmentation tasks with RSI. To better understand and stimulate future research that utilizes semantic segmentation tasks with small data, we summarized the supervised learning methods and challenges they face. We also reviewed the supervised approaches with data that are currently popular to help elucidate how to efficiently utilize a limited number of samples to address issues with semantic segmentation in RSI. The main methods discussed are self-supervised learning, semi-supervised learning, weakly supervised learning and few-shot methods. The solution of cross-domain challenges has also been discussed. Furthermore, multi-modal methods, prior knowledge constrained methods, and future research required to help optimize deep learning models for various downstream tasks in relation to RSI have been identified.

SatellStitch: Satellite Imagery-Assisted UAV Image Seamless Stitching for Emergency Response without GCP and GNSS

“…The first idea is to avoid the generation of gaps, and the optimal stitch line method is one of the representative methods [15], such as the snake model [16], Dijkstra's algorithm [17,18], the dynamic programming (DP) algorithm [19,20], graph cut algorithms [21,22] and ant colony optimization [23]. In addition, with the rapid development of deep learning convolutional neural networks (CNNs) over the last few years [24][25][26][27][28], Li et al [29] proposed to combine the semantic segmentation information of CNNs [30,31] to calculate the difference and search for the optimal stitching seam. By constructing different calculation rules, this method automatically searches for a stitch seam with the minimum difference in the image overlap region, which can effectively avoid the generation of gaps and has been adopted by many commercial software [32].…”

An Efficient Semantic Segmentation Method for Remote-Sensing Imagery Using Improved Coordinate Attention

“…Convolutional neural networks (CNNs) have strong learning abilities and can autonomously acquire rich spectral and spatial characteristics from images. Therefore, several scholars have successfully employed it in tasks related to remote-sensing image segmentation [10,11]. Different from traditional CNN, a full convolutional network (FCN) [12] can achieve pixellevel image classification.…”

Utilization of Improved Annotations From Object-Based Image Analysis as Training Data for DeepLab V3+ Model: A Focus on Road Extraction in Very High-Resolution Orthophotos

“…In contrast, deep learning (DL) excels in road extraction generation; however, it mandates road annotations during training, a task performed manually by operators, demanding significant time and labor resources for the provision of essential training data. [24], [25], [26], [27].…”