An Alternating Training Method of Attention-Based Adapters for Visual Explanation of Multi-Domain Satellite Images

Kim, Heejae; Lee, Kyungchae; Lee, Chang Ha; Hwang, Sang-Hyun; Youn, Chan‐Hyun

doi:10.1109/access.2021.3074640

Cited by 3 publications

(1 citation statement)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Applications like personalized service on mobile device [7], [8], anomaly detection from IoT device [9], and satellite imagery analysis [10] on on-board processing system [11] require deploying DL models on constrained computing environment. As one of use cases, Cloudscout [11] deploys the custom designed CNN for nanosatellite to select eligible data by detecting cloud as binary masking form.…”

Section: Related Work a DL Processing On Constrained Computing Enviro...mentioning

confidence: 99%

A Channel Pruning Optimization With Layer-Wise Sensitivity in a Single-Shot Manner Under Computational Constraints

et al. 2023

Self Cite

View full text Add to dashboard Cite

In the constrained computing environments such as mobile device or satellite on-board system, various computational factors of hardware resource can restrict the processing of deep learning (DL) services.Recent DL models such as satellite image analysis mainly require larger resource memory occupation for intermediate feature map footprint than the given memory specification of hardware resource and larger computational overhead (in FLOP) to meet service-level objective in the sense of hardware accelerator. As one of the solutions, we propose a new method of controlling the layer-wise channel pruning in a single-shot manner that can decide how much channels to prune in each layer by observing dataset once without full pretraining. To improve the robustness of the performance degradation, we also propose a layer-wise sensitivity and formulate the optimization problems for deciding layer-wise pruning ratio under target computational constraints. In the paper, the optimal conditions are theoretically derived, and the practical optimum searching schemes are proposed using the optimal conditions. On the empirical evaluation, the proposed methods show robustness on performance degradation, and present feasibility on DL serving under constrained computing environments by reducing memory occupation, providing acceleration effect and throughput improvement while keeping the accuracy performance.INDEX TERMS Single-shot pruning, channel pruning, lottery ticket hypothesis, DL model compression.

show abstract

Section: Related Work a DL Processing On Constrained Computing Enviro...mentioning

confidence: 99%

A Channel Pruning Optimization With Layer-Wise Sensitivity in a Single-Shot Manner Under Computational Constraints

et al. 2023

Self Cite

View full text Add to dashboard Cite

show abstract

Federated Onboard-Ground Station Computing With Weakly Supervised Cascading Pyramid Attention Network for Satellite Image Analysis

et al. 2022

Self Cite

View full text Add to dashboard Cite

Recursive Visual Explanations Mediation Scheme Based on DropAttention Model With Multiple Episodes Pool

Jeon

Kim

et al. 2023

IEEE Access

View full text Add to dashboard Cite

In some DL applications such as remote sensing, it is hard to obtain the high task performance (e.g. accuracy) using the DL model on image analysis due to the low resolution characteristics of the imagery. Accordingly, several studies attempted to provide visual explanations or apply the attention mechanism to enhance the reliability on the image analysis. However, there still remains structural complexity on obtaining a sophisticated visual explanation with such existing methods: 1) which layer will the visual explanation be extracted from, and 2) which layers the attention modules will be applied to. 3) Subsequently, in order to observe the aspects of visual explanations on such diverse episodes of applying attention modules individually, training cost inefficiency inevitably arises as it requires training the multiple models one by one in the conventional methods. In order to solve the problems, we propose a new scheme of mediating the visual explanations in a pixel-level recursively. Specifically, we propose DropAtt that generates multiple episodes pool by training only a single network once as an amortized model, which also shows stability on task performance regardless of layer-wise attention policy. From the multiple episodes pool generated by DropAtt, by quantitatively evaluating the explainability of each visual explanation and expanding the parts of explanations with high explainability recursively, our visual explanations mediatio scheme attempts to adjust how much to reflect each episodic layer-wise explanation for enforcing a dominant explainability of each candidate. On the empirical evaluation, our methods show their feasibility on enhancing the visual explainability by reducing average drop about 17% and enhancing the rate of increase in confidence 3%.INDEX TERMS Explainable AI (XAI), attention, class activation map (CAM), amortized model. I. INTRODUCTIONRecently, with the development of deep learning (DL) models, several studies [1]-[3] attempt to apply it on image analysis fields such as remote sensing or medical analysis. However, it is hard to clearly distinguish object classes on such satellite imagery due to its relatively low resolution, therefore, explanation on prediction is further required to provide reliability for the user via explainable AI (XAI) method [4], [5].

show abstract

An Alternating Training Method of Attention-Based Adapters for Visual Explanation of Multi-Domain Satellite Images

Cited by 3 publications

References 42 publications

A Channel Pruning Optimization With Layer-Wise Sensitivity in a Single-Shot Manner Under Computational Constraints

A Channel Pruning Optimization With Layer-Wise Sensitivity in a Single-Shot Manner Under Computational Constraints

Federated Onboard-Ground Station Computing With Weakly Supervised Cascading Pyramid Attention Network for Satellite Image Analysis

Recursive Visual Explanations Mediation Scheme Based on DropAttention Model With Multiple Episodes Pool

Contact Info

Product

Resources

About