Credible Remote Sensing Scene Classification Using Evidential Fusion on Aerial-Ground Dual-View Images

Abstract

Due to their ability to offer more comprehensive information than data from a single view, multi-view (e.g., multi-source, multi-modal, multi-perspective) data are being used more frequently in remote sensing tasks. However, as the number of views grows, the issue of data quality is becoming more apparent, limiting the potential benefits of multi-view data. Although recent deep neural network (DNN)-based models can learn the weight of data adaptively, a lack of research on explicitly quantifying the data quality of each view when fusing them renders these models inexplicable, performing unsatisfactorily and inflexibly in downstream remote sensing tasks. To fill this gap, in this paper, evidential deep learning is introduced to the task of aerial-ground dual-view remote sensing scene classification to model the credibility of each view. Specifically, the theory of evidence is used to calculate an uncertainty value which describes the decision-making risk of each view. Based on this uncertainty, a novel decision-level fusion strategy is proposed to ensure that the view with lower risk obtains more weight, making the classification more credible. On two well-known, publicly available datasets of aerial-ground dual-view remote sensing images, the proposed approach achieves state-of-the-art results, demonstrating its effectiveness.

Multi-View Scene Classification Based on Feature Integration and Evidence Decision Fusion

“…Unlike the traditional scene classification task, the loss function of MVEDFN is computed by integrating the Dirichlet distribution. To ensure that both perspectives can offer reasonable guidance for multi-view scene classification, the loss function of MVEDFN incorporates a hybrid approach by combining the aerial view Dirichlet distribution integral loss L loss α 1 i , the ground view Dirichlet distribution integral loss L loss α 2 i , and the fusion of multi-view with the Dirichlet distribution integral loss L loss α i f usion during network training [31], specifically, as shown in Equations ( 9) and ( 10):…”

“…The existing methods for the fusion classification of multi-view remote sensing scene images can be broadly classified into three levels: data level, feature level, and decision level. Therefore, the MVEDFN method proposed in this paper is compared with a data-level fusion method (the six-channel method [42]), two feature-level fusion methods (CILM [23] and MSAN [24]), and four decision-level fusion methods (SoftMax product [22], SoftMax sum [22], EFN [31], and TMC [41]). The following techniques are briefly outlined.…”

“…Decision-level fusion strategies, which summarize predictions from multiple classifiers using diverse fusion rules [28][29][30], have shown promise in multi-view classification [31]. This approach accommodates incorrect predictions from one perspective while still yielding accurate fault-tolerant results, even with poor image quality from one viewpoint [32].…”

“…Thus, there is a certain bias in intuitive cognition, and the fusion of aerial evidence and ground evidence requires better validity and adaptability and cannot provide a reasonable explanation for the final prediction results. For instance, Zhao et al [31] incorporated deep evidence learning theory in multi-view classification, but issues remain in terms of inadequate feature expression and unaddressed conflicts during evidence fusion.…”

Boosting Adversarial Transferability with Shallow-Feature Attack on SAR Images

“…Deep neural networks (DNNs) have achieved excellent performance in many remote sensing applications, such as object detection [4][5][6][7][8][9], image classification [10][11][12][13][14], and semantic segmentation [15][16][17][18][19][20]. However, DNNs have been shown by Szegedy [21] to be vulnerable to adversarial examples.…”