Detecting Potential Local Adversarial Examples for Human-Interpretable Defense

Abstract

Machine learning models are increasingly used in the industry to make decisions such as credit insurance approval. Some people may be tempted to manipulate specific variables, such as the age or the salary, in order to get better chances of approval. In this ongoing work, we propose to discuss, with a first proposition, the issue of detecting a potential local adversarial example on classical tabular data by providing to a human expert the locally critical features for the classifier's decision, in order to control the provided information and avoid a fraud.

Towards Design Principles for User-Centric Explainable AI in Fraud Detection

“…DP 1 states it is essential to enable fraud experts with the capabilities and limitations of AI. Probabilities provide an explicit visualization of AI models' confidence for predictions based on a trained dataset [47,32]. Moreover, the method of Adversarial explanations enables experts to assess the cases that would affect legitimate predictions.…”

Explanation Framework for Intrusion Detection

“…Namely, what we propose to call decision boundary centered explanations. While LIME illustrates which features contribute to an instance, Local Adversarial Detection (LAD) [4] and Local Surrogate [3] yield a feature attribution that is relevant at a local decision boundary. To do so, it is required to find the decision boundary first and then to train a surrogate on instances located around the decision boundary.…”

“…As mentioned in Section 2.2, this can be formulated as an optimization problem. Alternatively, we can use random approaches similar to [1] or [4]. Randomly sampling instances in a neighbourhood of the instance x 0 can be very expensive as the counterfactual might be far away in the feature space of possible instances.…”