"Is your explanation stable?"

Gan, Yuyou; Mao, Yuhao; Zhang, Xuhong; Ji, Shouling; Pu, Yuwen; Han, Meng; Yin, Jianwei; Wang, Ting

doi:10.1145/3548606.3559392

Cited by 11 publications

(5 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hsieh et al [28] propose Robustness-S to evaluate explanation methods and design a new search-based explanation method, Greedy-AS. Gan et al [18] propose the Median Test for Feature Attribution to evaluate and improve the robustness of explanation methods. Traditional tests are used in the paper, which may also suffer from the random dominance problem.…”

Section: Robustness Of Explanation Methodsmentioning

confidence: 99%

“…Based on previous studies, stable explanations ensure that if a given input is perturbed within 𝜖 and the model's output label remains unchanged, the corresponding explanations will stay stable [17,18,28]. However, stable explanations may not always guarantee faithfulness [18], as stable and faithful are two different properties of explanations. There could be the cases where explanations are stable but not faithful.…”

Section: E Adversarial Attack On Explanation Methodsmentioning

confidence: 99%

“…However, it is non-trivial to evaluate the faithfulness of an explanation method, as indicated in Figure 1 and 2. The formal definition of faithfulness varies across studies [18,22]. In this section, we first review the definition of the traditional faithfulness tests from previous work and then present our new trend tests in the next section.…”

Section: Problem Definitionmentioning

confidence: 99%

“…There are three common tests for the local explanation, i.e., synthesis test, augmentation test and reduction test [22,66]. These tests are widely used as SOTA methods in recent research [17,18]. The intuition of these tests is to modify an input guided by explanation results and observe the change of the target label's posterior probability by the model, i.e., F 𝑡𝑎𝑟𝑔𝑒𝑡 (X).…”

Section: Problem Definitionmentioning

confidence: 99%

See 3 more Smart Citations

Good-looking but Lacking Faithfulness: Understanding Local Explanation Methods through Trend-based Testing

He,

Chen,

Meng

et al. 2023

Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security

View full text Add to dashboard Cite

While enjoying the great achievements brought by deep learning (DL), people are also worried about the decision made by DL models, since the high degree of non-linearity of DL models makes the decision extremely difficult to understand. Consequently, attacks such as adversarial attacks are easy to carry out, but difficult to detect and explain, which has led to a boom in the research on local explanation methods for explaining model decisions. In this paper, we evaluate the faithfulness of explanation methods and find that traditional tests on faithfulness encounter the random dominance problem, i.e., the random selection performs the best, especially for complex data. To further solve this problem, we propose three trend-based faithfulness tests and empirically demonstrate that the new trend tests can better assess faithfulness than traditional tests on image, natural language and security tasks. We implement the assessment system and evaluate ten popular explanation methods. Benefiting from the trend tests, we successfully assess the explanation methods on complex data for the first time, bringing unprecedented discoveries and inspiring future research. Downstream tasks also greatly benefit from the tests. For example, model debugging equipped with faithful explanation methods performs much better for detecting and correcting accuracy and security problems. CCS CONCEPTS• Security and privacy → Software and application security.

show abstract

Section: Robustness Of Explanation Methodsmentioning

confidence: 99%

Section: E Adversarial Attack On Explanation Methodsmentioning

confidence: 99%

Section: Problem Definitionmentioning

confidence: 99%

Section: Problem Definitionmentioning

confidence: 99%

See 2 more Smart Citations

Good-looking but Lacking Faithfulness: Understanding Local Explanation Methods through Trend-based Testing

He,

Chen,

Meng

et al. 2023

Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security

View full text Add to dashboard Cite

show abstract

“…Pandey [22] adopted LSTM as an encoder to achieve machine translation, which improved the effectiveness of machine translation. However, the autoencoder network based on the attention mechanism has only been proven effective in machine translation, face recognition, and image processing, and there are few studies on time series prediction [23][24][25].…”

Section: Introductionmentioning

confidence: 99%

RAdam-DA-NLSTM: A Nested LSTM-Based Time Series Prediction Method for Human–Computer Intelligent Systems

et al. 2023

Self Cite

View full text Add to dashboard Cite

At present, time series prediction methods are widely applied for Human–Computer Intelligent Systems in various fields such as Finance, Meteorology, and Medicine. To enhance the accuracy and stability of the prediction model, this paper proposes a time series prediction method called RAdam-Dual stage Attention mechanism-Nested Long Short-Term Memory (RAdam-DA-NLSTM). First, we design a Nested LSTM (NLSTM), which adopts a new internal LSTM unit structure as the memory cell of LSTM to guide memory forgetting and memory selection. Then, we design a self-encoder network based on the Dual stage Attention mechanism (DA-NLSTM), which uses the NLSTM encoder based on the input attention mechanism, and uses the NLSTM decoder based on the time attention mechanism. Additionally, we adopt the RAdam optimizer to solve the objective function, which dynamically selects Adam and SGD optimizers according to the variance dispersion and constructs the rectifier term to fully express the adaptive momentum. Finally, we use multiple datasets, such as PM2.5 data set, stock data set, traffic data set, and biological signals, to analyze and test this method, and the experimental results show that RAdam-DA-NLSTM has higher prediction accuracy and stability compared with other traditional methods.

show abstract