The Dangers of Post-hoc Interpretability: Unjustified Counterfactual Explanations

Laugel, Thibault; Lesot, Marie‐Jeanne; Marsala, Christophe; Renard, Xavier; Detyniecki, Marcin

doi:10.48550/arxiv.1907.09294

Cited by 21 publications

(29 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The advantages of our algorithm are twofold: (1) it can guarantee sample acceptance in high dimensional space where the rejection sampling based on the Monte Calro method easily fails when the region area is unknown; (2) it can handle sampling strategy in a perspective of the model where the commonly used -based sampling (Erhan, Courville, and Bengio 2010) is not precise to obtain samples considering complex non-spherical generative boundaries ( Laugel et al 2019). We experimentally verify that our algorithm obtains more consistent samples compared to -based sampling methods on deep convolutional GANs (DCGAN) (Radford, Metz, and Chintala 2015) and progressive growing of GANs (PGGAN) (Karras, Aila, and Laine 2015).…”

Section: Introductionmentioning

confidence: 99%

An Efficient Explorative Sampling Considering the Generative Boundaries of Deep Generative Neural Networks

Jeon

Jeong

Choi

2020

AAAI

View full text Add to dashboard Cite

Deep generative neural networks (DGNNs) have achieved realistic and high-quality data generation. In particular, the adversarial training scheme has been applied to many DGNNs and has exhibited powerful performance. Despite of recent advances in generative networks, identifying the image generation mechanism still remains challenging. In this paper, we present an explorative sampling algorithm to analyze generation mechanism of DGNNs. Our method efficiently obtains samples with identical attributes from a query image in a perspective of the trained model. We define generative boundaries which determine the activation of nodes in the internal layer and probe inside the model with this information. To handle a large number of boundaries, we obtain the essential set of boundaries using optimization. By gathering samples within the region surrounded by generative boundaries, we can empirically reveal the characteristics of the internal layers of DGNNs. We also demonstrate that our algorithm can find more homogeneous, the model specific samples compared to the variations of ϵ-based sampling method.

show abstract

Section: Introductionmentioning

confidence: 99%

An Efficient Explorative Sampling Considering the Generative Boundaries of Deep Generative Neural Networks

Jeon

Jeong

Choi

2020

AAAI

View full text Add to dashboard Cite

show abstract

“…A similar desideratum is considered in [48] and [52]; the latter work employing neural autoencoders to that end. Laugel et al [35,36] require that z can always be reached from a training point x ′ without having to cross the decision boundary of 𝑓 , for z not to be the result of an artifact in the decision boundary of 𝑓 . In [26] and [41], counterfactual explanations are studied through the lens of causality.…”

Section: Related Workmentioning

confidence: 99%

“…Lastly, Recidivism (Rec) is a data set collected by an investigation of ProPublica about possible racial bias in the commercial software COMPAS, which intends to estimate the risk that an inmate will re-offend [33]. Examples of recent works in fair and explainable machine learning that adopted (some of) these data sets (each) are [8,9,17,28,32,35,54].…”

Section: Data Setsmentioning

confidence: 99%

On the Robustness of Sparse Counterfactual Explanations to Adverse Perturbations

Virgolin¹,

Fracaros²

2022

Preprint

View full text Add to dashboard Cite

Counterfactual explanations (CEs) are a powerful means for understanding how decisions made by algorithms can be changed.Researchers have proposed a number of desiderata that CEs should meet to be practically useful, such as requiring minimal effort to enact, or complying with causal models. We consider a further aspect to improve the usability of CEs: robustness to adverse perturbations, which may naturally happen due to unfortunate circumstances. Since CEs typically prescribe a sparse form of intervention (i.e., only a subset of the features should be changed), we provide two definitions of robustness, which concern, respectively, the features to change and to keep as they are. These definitions are workable in that they can be incorporated as penalty terms in the loss functions that are used for discovering CEs. To experiment with the proposed definitions of robustness, we create and release code where five data sets (commonly used in the field of fair and explainable machine learning) have been enriched with feature-specific annotations that can be used to sample meaningful perturbations. Our experiments show that CEs are often not robust and, if adverse perturbations take place, the intervention they prescribe may require a much larger cost than anticipated, or even become impossible. However, accounting for robustness in the search process, which can be done rather easily, allows discovering robust CEs systematically.

show abstract

“…Justified Counterfactual Explanations In 2019, Laungel et al [27], for classification problems, discussed the risk of making counterfactual explanations based on artifacts learned by the model, instead of actual knowledge. This problem arises for instances that lie in a subspace where there is no reliable information.…”

Section: B Contrastive Approachmentioning

confidence: 99%

Individual Explanations in Machine Learning Models: A Survey for Practitioners

Carrillo,

Cantú,

Noriega

2021

Preprint

View full text Add to dashboard Cite

In recent years, the use of sophisticated statistical models that influence decisions in domains of high societal relevance is on the rise. Although these models can often bring substantial improvements in the accuracy and efficiency of organizations, many governments, institutions, and companies are reluctant to their adoption as their output is often difficult to explain in human-interpretable ways. Hence, these models are often regarded as black-boxes, in the sense that their internal mechanisms can be opaque to human audit. In real-world applications, particularly in domains where decisions can have a sensitive impact-e.g., criminal justice, estimating credit scores, insurance risk, health risks, etc.-model interpretability is desired. Recently, the academic literature has proposed a substantial amount of methods for providing interpretable explanations to machine learning models. This survey reviews the most relevant and novel methods that form the state-of-the-art for addressing the particular problem of explaining individual instances in machine learning. It seeks to provide a succinct review that can guide data science and machine learning practitioners in the search for appropriate methods to their problem domain.

show abstract

The Dangers of Post-hoc Interpretability: Unjustified Counterfactual Explanations

Cited by 21 publications

References 0 publications

An Efficient Explorative Sampling Considering the Generative Boundaries of Deep Generative Neural Networks

An Efficient Explorative Sampling Considering the Generative Boundaries of Deep Generative Neural Networks

On the Robustness of Sparse Counterfactual Explanations to Adverse Perturbations

Individual Explanations in Machine Learning Models: A Survey for Practitioners

Contact Info

Product

Resources

About