An End-to-End Conditional Random Fields and Skip-Connected Generative Adversarial Segmentation Network for Remote Sensing Images

Abstract

Semantic segmentation is an important process of scene recognition with deep learning frameworks achieving state of the art results, thus gaining much attention from the remote sensing community. In this paper, an end-to-end conditional random fields generative adversarial segmentation network is proposed. Three key factors of this algorithm are as follows. First, the network combines generative adversarial network and Bayesian framework to realize the estimation from the prior probability to the posterior probability. Second, the skip connected encoder-decoder network is combined with CRF layer to implement end-to-end network training. Finally, the adversarial loss and the cross-entropy loss guide the training of the segmentation network through back propagation. The experimental results show that our proposed method outperformed FCN in terms of mIoU for 0.0342 and 0.11 on two data sets, respectively.

Urban Land Cover Classification of High-Resolution Aerial Imagery Using a Relation-Enhanced Multiscale Convolutional Network

“…Some studies make CRFs differentiable [39] or turn them into a recurrent network [40] to train with the network end-to-end. He et al [41] integrated a skip-connected encoder-decoder network structure and CRF layer to implement end-to-end network training, and the result was improved by taking more information into account. To reduce salt-and-pepper noise, Zhao et al [11] transformed image pixel labels into semantic segments and presented a semantic segment-based CRF method to effectively exploit the contextual relationships between different categories of ground objects.…”

An End-To-End Bayesian Segmentation Network Based on a Generative Adversarial Network for Remote Sensing Images

“…Moreover, the term abs means to take the absolute value of the amount in the associated brackets. In the paper [35], g loss contains a cross-entropy loss term, an L1 loss term, and an adversarial loss term g GAN . From Equations 3and 6, it can be found that the L1 loss calculates the absolute difference between the labels, and the cross-entropy is the degree of correlation between the label probabilities.…”

“…Luc et al [28] first used GAN to image segmentation for the purpose of enhancing long-range spatial label contiguity without increasing the complexity of the model used in the test. Subsequently, a variety of GAN-based segmentation methods [29][30][31][32][33][34][35] were proposed. Zhu et al [29] proposed an end-to-end network with the trained adversarial deep structure to improve the robustness of a small data model and prevent over-fitting.…”

Improved Winter Wheat Spatial Distribution Extraction Using A Convolutional Neural Network and Partly Connected Conditional Random Field

“…To complete the calculations, previous studies have used approximate calculations [60,61], reduction of the number of samples involved in modeling [62,63], and introduced conditional independence [64][65][66]. However, in doing so, the performance of the CRFs gets reduced [67]. To combine a CNN and CRFs, and achieve end-to-end training, several studies [67][68][69] have converted the CRF into an iterative calculation, while others [64] have converted the CRF into a convolution operation.…”

“…However, in doing so, the performance of the CRFs gets reduced [67]. To combine a CNN and CRFs, and achieve end-to-end training, several studies [67][68][69] have converted the CRF into an iterative calculation, while others [64] have converted the CRF into a convolution operation.…”