Knowledge Based Descriptive Neural Networks

Abstract

This paper presents a study of knowledge based descriptive neural networks (DNN). DNN is a neural network that incorporates rules extracted from trained neural networks. One of the major drawbacks of neural network models is that they could not explain what they have done. Extracting rules from trained neural networks is one of the solutions. However, how to effectively use extracted rules has been paid little attention. This paper addresses issues of effective ways of using these extracted rules. With the introduction of DNN, we not only keep the good feature of nonlinearity in neural networks but also have explanation of underlying reasoning mechanisms, for instance, how prediction is made.

A multi-output descriptive neural network for estimation of scour geometry downstream from hydraulic structures

“…In the third stage, the formulations extracted from the previous stage is incorporated to the network generated by first stage to form a DNN. Researchers genarally use if-then type rules association rules [34]. In this study, the information hidden in a well-trained NN is extracted as linear regression equations embedded in input layer and logistic functions embedded in the output layer of the NN architecture.…”

“…However, once an NN model is trained for its generalization properties, it can be demonstrated that the trained model represents the physical process of the system. The knowledge acquired for the problem domain during the training process is encoded within the NN in two forms: (a) in the network architecture itself (through number of hidden units), and: (b) in a set of constants or weights [34]. Lange [24] states that ANNs are black-box models that only develop the relation between input and output variables without the modelling of any physical processes, however, it must be realized that the data that are employed in developing black-box models contain important information about the physical processes being modeled, and this information gets embedded or captured inside the model [9].…”

“…Lange [24] states that ANNs are black-box models that only develop the relation between input and output variables without the modelling of any physical processes, however, it must be realized that the data that are employed in developing black-box models contain important information about the physical processes being modeled, and this information gets embedded or captured inside the model [9]. Although there are several attempts in other scientific branches that have shown that useful information could be obtained from trained neural networks (see Yao [34] for relevant references), limited research has been done for water-engineering applications [3,9,16,18,19].…”

Knowledge discovery in corporate events by neural network rule extraction

“…On the other hand, only a few methods have been developed to extract rules from trained networks for regression problems [14,24,25]. Recently, Yao [26] addressed the issue of using the discovered underlying rules by neural networks, and presented an ongoing project of incorporating rules extracted from trained neural networks to form a knowledge-based descriptive neural network (DNN). In neural network rule extraction, fidelity describes how well the rules mimic the behavior of a neural network while accuracy describes how well the rules can be generalized.…”

Knowledge extracted from trained neural networks: What's next?

“…It was argued that by incorporating extracted rules with a neural network to reconstruct a descriptive neural network may help us to gain more understanding of neural network mechanisms and make more accurate forecastings. 25 It was reported that many researchers confuse two different goals of studies of rule extraction from neural networks. 30 The first goal is to obtain accurate and comprehensible learning systems, and the second goal is understanding the working mechanism of neural networks.…”