Efficiently utilizing complex-valued PolSAR image data via a multi-task deep learning framework

Zhang, Lamei; Dong, Hongbiao; Zou, Bin

doi:10.1016/j.isprsjprs.2019.09.002

Cited by 25 publications

(15 citation statements)

References 66 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, the extracted features were classified by layers in the back of the network. In addition, there were some other deep networks used in the classification of PolSAR image, such as 3D-CNN combined with the conditional random field [30], CNN combined with graph [31], polarimatric convolutional network [32], Wishart deep stacking network S [33], Wishart autoencoder (Wishart-AE) [34], Wishart convolutional autoencoder (Wishart-CAE) [34], and the improved deep stacked network [35], [36]. Because some superpixel segmentation methods were proved to be effective in preserving spatial structure information of PolSAR images [37]- [39], the superpixel-based graph convolutional network was also proposed for PolSAR image classification [40].…”

Section: Introductionmentioning

confidence: 99%

A Lightweight Complex-Valued DeepLabv3+ for Semantic Segmentation of PolSAR Image

Zeng

Liu

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Semantic image segmentation is one kind of end-to-end segmentation methods which can classify the target region pixel by pixel. As a classic semantic segmentation network in optical images, DeepLabv3+ can achieve a good segmentation performance. However, when this network is directly used in the semantic segmentation of polarimetric synthetic aperture radar (PolSAR) image, it is hard to obtain the ideal segmentation results. The reason is that it is easy to yield overfitting due to the small PolSAR dataset. In this paper, a lightweight complex-valued DeepLabv3+ (L-CV-DeepLabv3+) is proposed for semantic segmentation of PolSAR data. It has two significant advantages, when compared with the original DeepLabv3+. First, the proposed network with the simplified structure and parameters can be suitable for the small PolSAR data, and thus it can effectively avoid the overfitting. Second, the proposed complex-valued (CV) network can make full use of both amplitude and phase information of PolSAR data, which brings better segmentation performance than the real-valued (RV) network, and the related CV operations are strictly true in the mathematical sense. Experimental results about two Flevoland datasets and one San Francisco dataset show that the proposed network can obtain better overall average (OA), mean intersection over union (MIOU), and mean pixel accuracy (MPA) than the original DeepLabv3+ and some other RV semantic segmentation networks.

show abstract

Section: Introductionmentioning

confidence: 99%

A Lightweight Complex-Valued DeepLabv3+ for Semantic Segmentation of PolSAR Image

Zeng

Liu

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…To avoid complex operations, Liu et al attempted to learn the feature of phase independently [27]. A two-stream architecture was proposed to extract features from amplitude and phase respectively with the aid of a multi-task feature fusion mechanism [28]. It is worth noting that PolSAR covariance matrix has been used as the input of CNNs in most studies [12,15,16,23,28], and the phase information is hidden between the input channels when each element of the upper triangle of PolSAR covariance matrix is regarded as a channel of the input.…”

Section: Introductionmentioning

confidence: 99%

“…A two-stream architecture was proposed to extract features from amplitude and phase respectively with the aid of a multi-task feature fusion mechanism [28]. It is worth noting that PolSAR covariance matrix has been used as the input of CNNs in most studies [12,15,16,23,28], and the phase information is hidden between the input channels when each element of the upper triangle of PolSAR covariance matrix is regarded as a channel of the input. Recent works have revealed that, with the aid of 3D operations, channel-wise correlations can be plugged in as an additional dimension of convolution kernels to solve the problem of feature mining on special data (e.g.…”

Section: Introductionmentioning

confidence: 99%

PolSAR Image Classification with Lightweight 3D Convolutional Networks

2020

Self Cite

View full text Add to dashboard Cite

Convolutional neural networks (CNNs) have become the state-of-the-art in optical image processing. Recently, CNNs have been used in polarimetric synthetic aperture radar (PolSAR) image classification and obtained promising results. Unlike optical images, the unique phase information of PolSAR data expresses the structure information of objects. This special data representation makes 3D convolution which explicitly modeling the relationship between polarimetric channels perform better in the task of PolSAR image classification. However, the development of deep 3D-CNNs will cause a huge number of model parameters and expensive computational costs, which not only leads to the decrease of the interpretation speed during testing, but also greatly increases the risk of over-fitting. To alleviate this problem, a lightweight 3D-CNN framework that compresses 3D-CNNs from two aspects is proposed in this paper. Lightweight convolution operations, i.e., pseudo-3D and 3D-depthwise separable convolutions, are considered as low-latency replacements for vanilla 3D convolution. Further, fully connected layers are replaced by global average pooling to reduce the number of model parameters so as to save the memory. Under the specific classification task, the proposed methods can reduce up to 69.83% of the model parameters in convolution layers of the 3D-CNN as well as almost all the model parameters in fully connected layers, which ensures the fast PolSAR interpretation. Experiments on three PolSAR benchmark datasets, i.e., AIRSAR Flevoland, ESAR Oberpfaffenhofen, EMISAR Foulum, show that the proposed lightweight architectures can not only maintain but also slightly improve the accuracy under various criteria.

show abstract

“…slow speed and lack of global information, fully convolutional networks [26] was introduced to achieve better full image classification [27], [28]. Some works studied new representations of PolSAR data, so as to improve the performance of CNN-based methods from the perspective of data [29]- [31]. Considering the particularity of PolSAR data, complex-valued CNN architectures have also been studied to some extent [32]- [34].…”

mentioning

confidence: 99%

“…This difficulty is partly reflected in the fact that architecture design requires considerable expertise of neural networks, which is lacking in PolSAR area. However, the author believes that with the efforts of researchers [31], [32], this will not be a stumbling block for future development. The author thinks that the main problem in the future research of CNNs-based PolSAR image classification methods is that the unsatisfactory performance of hand-crafted suboptimal architectures.…”

mentioning

confidence: 99%

Automatic Design of CNNs via Differentiable Neural Architecture Search for PolSAR Image Classification

Dong

Zou

Zhang

et al. 2020

IEEE Trans. Geosci. Remote Sensing

Self Cite

View full text Add to dashboard Cite

Convolutional neural networks (CNNs) have shown good performance in polarimetric synthetic aperture radar (PolSAR) image classification due to the automation of feature engineering. Excellent hand-crafted architectures of CNNs incorporated the wisdom of human experts, which is an important reason for CNN's success. However, the design of the architectures is a difficult problem, which needs a lot of professional knowledge as well as computational resources. Moreover, the architecture designed by hand might be suboptimal, because it is only one of thousands of unobserved but objective existed paths. Considering that the success of deep learning is largely due to its automation of the feature engineering process, how to design automatic architecture searching methods to replace the handcrafted ones is an interesting topic. In this paper, we explore the application of neural architecture search (NAS) in PolSAR area for the first time. Different from the utilization of existing NAS methods, we propose a differentiable architecture search (DAS) method which is customized for PolSAR classification. The proposed DAS is equipped with a PolSAR tailored search space and an improved one-shot search strategy. By DAS, the weights parameters and architecture parameters (corresponds to the hyperparameters but not the topologies) can be optimized by stochastic gradient descent method during the training. The optimized architecture parameters should be transformed into corresponding CNN architecture and re-train to achieve highprecision PolSAR classification. In addition, complex-valued DAS is developed to take into account the characteristics of PolSAR images so as to further improve the performance. Experiments on three PolSAR benchmark datasets show that the CNNs obtained by searching have better classification performance than the hand-crafted ones. He is currently a Professor and Vice Head with the Department of Information Engineering, Harbin Institute of Technology. His research interests include SAR image processing, polarimetric SAR, and polarimetric SAR interferometry.

show abstract

Efficiently utilizing complex-valued PolSAR image data via a multi-task deep learning framework

Cited by 25 publications

References 66 publications

A Lightweight Complex-Valued DeepLabv3+ for Semantic Segmentation of PolSAR Image

A Lightweight Complex-Valued DeepLabv3+ for Semantic Segmentation of PolSAR Image

PolSAR Image Classification with Lightweight 3D Convolutional Networks

Automatic Design of CNNs via Differentiable Neural Architecture Search for PolSAR Image Classification

Contact Info

Product

Resources

About