Probing Model Signal-Awareness via Prediction-Preserving Input Minimization

Machine learning (ML) models play an increasingly prevalent role in many software engineering tasks. However, because most models are now powered by opaque deep neural networks, it can be difficult for developers to understand why the model came to a certain conclusion and how to act upon the model's prediction. Motivated by this problem, this paper explores counterfactual explanations for models of source code. Such counterfactual explanations constitute minimal changes to the source code under which the model "changes its mind". We integrate counterfactual explanation generation to models of source code in a real-world setting. We describe considerations that impact both the ability to find realistic and plausible counterfactual explanations, as well as the usefulness of such explanation to the developers that use the model. In a series of experiments we investigate the efficacy of our approach on three different models, each based on a BERT-like architecture operating over source code.

show abstract

Section: Interpretability Of Se Modelsmentioning

confidence: 99%

Counterfactual Explanations for Models of Code

Cito

Dillig

Murali

et al. 2022

2022 IEEE/ACM 44th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP)

View full text Add to dashboard Cite

show abstract

“…Interpretability of SE models. There has also been recent interest in improving interpretability of models used in software engineering [6,8,31,41]. Two of these efforts [31,41] propose to simplify the code while retaining the model prediction.…”

Section: Related Workmentioning

confidence: 99%

“…There has also been recent interest in improving interpretability of models used in software engineering [6,8,31,41]. Two of these efforts [31,41] propose to simplify the code while retaining the model prediction. Another effort called AutoFocus [6] aims to rate and visualize the relative importance of different code elements by using a combination of attention layers in the neural network and deleting statements in the program.…”

Section: Related Workmentioning

confidence: 99%

Counterfactual Explanations for Models of Code

Cito¹,

Dillig²,

Murali³

et al. 2021

Preprint

View full text Add to dashboard Cite

Machine learning (ML) models play an increasingly prevalent role in many software engineering tasks. However, because most models are now powered by opaque deep neural networks, it can be difficult for developers to understand why the model came to a certain conclusion and how to act upon the model's prediction. Motivated by this problem, this paper explores counterfactual explanations for models of source code. Such counterfactual explanations constitute minimal changes to the source code under which the model "changes its mind". We integrate counterfactual explanation generation to models of source code in a real-world setting. We describe considerations that impact both the ability to find realistic and plausible counterfactual explanations, as well as the usefulness of such explanation to the user of the model. In a series of experiments we investigate the efficacy of our approach on three different models, each based on a BERT-like architecture operating over source code.

show abstract

Probing Model Signal-Awareness via Prediction-Preserving Input Minimization

Cited by 2 publications

References 0 publications

Counterfactual Explanations for Models of Code

Counterfactual Explanations for Models of Code

Counterfactual Explanations for Models of Code

Contact Info

Product

Resources

About