Rule Extraction from Training Data Using Neural Network

Abstract

Data Mining is a powerful technology to help organization to concentrate on most important data by extracting useful information from large database. One of the most commonly used techniques in data mining is Artificial Neural Network due to its high performance in many application domains. Despite many advantages of Artificial Neural Network, one of its main drawbacks is its inherent black box nature which is the main problem of using Artificial Neural Network in data mining. Therefore, this paper proposes a rule extraction algorithm from neural network using classified and misclassified data to convert the black box nature of Artificial Neural Network into a white box. The proposed algorithm is a modification of the existing algorithm, Rule Extraction by Reverse Engineering (RxREN). The proposed algorithm extracts rules from trained neural network for datasets with mixed mode attributes using pedagogical approach. The proposed algorithm uses both classified as well as misclassified data to find out the data ranges of significant attributes in respective classes, which is the innovation of the proposed algorithm. The experimental results clearly show that the performance of the proposed algorithm is superior to existing algorithms.

A Survey on the Explainability of Supervised Machine Learning

“…This measure is then used to build rule conditions. The measure determines the features and the features' values of a condition (Biswas et al, 2017).…”

“…This is done by applying an approach similar to sensitivity analysis. The Binarized Input-Output Rule Extraction (BIO-RE) (Biswas et al, 2017) algorithm applies a sampling based approach that generates all possible input combinations and asks the black box for their predictions. Based on that, a truth table is build on top of which an arbitrary rule-based algorithm can be applied to extract rules.…”

“…Samek et al (2017) describe different methods for visualizing and explaining deep learning models like the Sensitivity Analysis (SA) or the Layer-wise relevance propagation (LRP). Biswas et al (2017) or Zilke et al (2016) also mention different techniques for decompositional decision rules from ANNs. Since the focus of this paper lies on explanations for SML mainly for tabular data, only a few explanation methods for deep learning models are mentioned.…”

Semantically Explainable Fuzzy Classifier

“…In the case of NNs, it means that rules are extracted at the level of individual units (hidden and output units), which are then combined to form rules. 5 RX 27 and Recursive-Rule-eXtraction (Re-RX) 28 algorithms recursively generate rules by analyzing activations of hidden units of a network with one hidden layer. To achieve higher rule recognition rates, an Ensemble-Recursive-Rule-eXtraction (E-Re-RX) 12 algorithm uses two and algorithm proposed in Ref.…”

Weighted Fuzzy Production Rule Extraction Using Modified Harmony Search Algorithm and BP Neural Network Framework

“…Likewise, Amit Gupta et al proposed another classification method based on the characteristics of some existing extraction rule algorithms in the "Input Network Structure Output Extraction Knowledge" stage. They classified the rule extraction algorithms into two categories, one is the generation and detection method, and the other is the analysis methods [11,12].…”