The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks

Abstract

To date, the preponderance of techniques for eliciting the knowledge embedded in trained artificial neural networks (ANN's) has focused primarily on extracting rule-based explanations from feedforward ANN's. The ADT taxonomy for categorizing such techniques was proposed in 1995 to provide a basis for the systematic comparison of the different approaches. This paper shows that not only is this taxonomy applicable to a cross section of current techniques for extracting rules from trained feedforward ANN's but also how the taxonomy can be adapted and extended to embrace a broader range of ANN types (e.g., recurrent neural networks) and explanation structures. In addition the paper identifies some of the key research questions in extracting the knowledge embedded within ANN's including the need for the formulation of a consistent theoretical basis for what has been, until recently, a disparate collection of empirical results.

Destructive Algorithm for Rule Extraction based on a Trained Neural Network

“…A classification of the rule extraction algorithms from neural network may characterize different methods using five dimensions [35]: (a) the "expressive power" of the extracted rules (types of rules extracted); (b) the "quality" of the extracted rules (accuracy, fidelity comparing to the underlying network, comprehensibility and consistency of the extracted rules); (c) the "translucency" of the method, based on localglobal use of the neural network (analysis of individual nodes versus analysis of the total network function); (d) the algorithmic complexity of the method; (e) specialized network training schemes. One should add one more dimension to this scheme, (f) the treatment of linguistic variables: some methods work only with binary variables, other with discretized inputs, and yet other with continuous variables that are converted to linguistic variables automatically.…”

X-TREPAN: An Extended Trepan for Comprehensibility and Classification Accuracy in Artificial Neural Networks

“…The ability to extract symbolic knowledge has many potential advantages: the knowledge obtained from the neural network can lead to new insights into patterns and dependencies within the data; from symbolic knowledge, it is easier to see which features of the data are the most important; and the explanation of a decision is essential for many applications, such as safety critical systems. Andrews et al and Ticke et al [9], [10] summarize several proposed approaches to rule extraction. Many of the earlier approaches required a specialized neural network architectures or training schemes.…”

“…Neural networks generate their own implicit rules by learning from examples. Artificial neural networks have been applied to a variety of problem domains [1] such as medical diagnostics [2], games [3], robotics [4], speech generation [5] and speech recognition [6]. The generalization ability of neural networks has proved to be superior to other learning systems over a wide range of applications [7].…”

Knowledge Based Descriptive Neural Networks

“…The fifth criterion of taxonomy of Tickle et al [14] is to measure the generalization ability of the technique. This dimension can be extended to applications as well.…”

“…According to the taxonomy of Tickle et al [14], neural network rule extraction techniques may be classified into five dimensions, namely, (1) the "expressive power" of the extracted rules (format or type of rules extracted); (2) the "quality" (accuracy, fidelity, consistency and comprehensibility) of the extracted rules; (3) the "translucency" of the view taken within the rule extraction technique of the underlying network units (i.e. using of decompositional or pedagogical); (4) the complexity of the algorithms; (5) the portability to network architectures and training regimes.…”