Dynamic Convolution Self-Attention Network for Land-Cover Classification in VHR Remote-Sensing Images

Abstract

The current deep convolutional neural networks for very-high-resolution (VHR) remote-sensing image land-cover classification often suffer from two challenges. First, the feature maps extracted by network encoders based on vanilla convolution usually contain a lot of redundant information, which easily causes misclassification of land cover. Moreover, these encoders usually require a large number of parameters and high computational costs. Second, as remote-sensing images are complex and contain many objects with large-scale variances, it is difficult to use the popular feature fusion modules to improve the representation ability of networks. To address the above issues, we propose a dynamic convolution self-attention network (DCSA-Net) for VHR remote-sensing image land-cover classification. The proposed network has two advantages. On one hand, we designed a lightweight dynamic convolution module (LDCM) by using dynamic convolution and a self-attention mechanism. This module can extract more useful image features than vanilla convolution, avoiding the negative effect of useless feature maps on land-cover classification. On the other hand, we designed a context information aggregation module (CIAM) with a ladder structure to enlarge the receptive field. This module can aggregate multi-scale contexture information from feature maps with different resolutions using a dense connection. Experiment results show that the proposed DCSA-Net is superior to state-of-the-art networks due to higher accuracy of land-cover classification, fewer parameters, and lower computational cost. The source code is made public available.

Common Practices and Taxonomy in Deep Multiview Fusion for Remote Sensing Applications

“…To illustrate, input and feature fusions (see Fig. 3 at bottom-left) [72], [74]- [80], or feature and decision fusions (see Fig. 3 at bottom-right) [33], [58], [81] are used together, improving the prediction performance compared to using a single layer fusion.…”

“…Nevertheless, in RS image-based applications, the use-cases have been extended as a way to enhance the information. Then, attention has been applied across spatial [100], [177], spectral [123], [134], [181], spatio-spectral [78], [80], [136], [139], or vector [138] dimensions. The motivation is that attention mechanisms lead to adaptively enhance the most relevant features of the data.…”

“…by using SAR view [1], [15], [33], [34], [36], [99], [113], [120], [124], [126], [143], [163], [173], [174], [188], [190], [193], [199]- [202] or LiDAR view [94], [139]. Furthermore, the DSM view has been widely used together with the optical view [17]- [19], [55], [58], [64], [65], [76], [80], [84], [85], [88], [89], [92], [104], [123], [184], where on some occasions is a LiDAR-derived DSM [32], [34], [35], [59], [62], [81], [86], [87], [93], [96], [133], …”