This work describes a methodology to extract symbolic rules from trained neural networks. In our approach, patterns on the network are codified using formulas on a Lukasiewicz logic. For this we take advantage of the fact that every connective in this multi-valued logic can be evaluated by a neuron in an artificial network having, by activation function the identity truncated to zero and one. This fact simplifies symbolic rule extraction and allows the easy injection of formulas into a network architecture. We trained this type of neural network using a back-propagation algorithm based on LevendergMarquardt algorithm, where in each learning iteration, we restricted the knowledge dissemination in the network structure. This makes the descriptive power of produced neural networks similar to the descriptive power of Lukasiewicz logic language, minimizing the information loss on the translation between connectionist and symbolic structures. To avoid redundance on the generated network, the method simplifies them in a pruning phase, using the "Optimal Brain Surgeon" algorithm. We tested this method on the task of finding the formula used on the generation of a given truth table. For real data tests, we selected the Mushrooms data set, available on the UCI Machine Learning Repository.
No abstract
This work describes a methodology that combines logic-based systems and connectionist systems. Our approach uses finite truth-valued Łukasiewicz logic, where we take advantage of fact, presented in (Castro and Trillas, 1998), wherein every connective can be defined by a neuron in an artificial network having, by activation function, the identity truncated to zero and one. This allowed the injection of formulas into a network architecture, and also simplified symbolic rule extraction. Neural networks are trained using the Levenderg-Marquardt algorithm, where we restricted the knowledge dissemination in the network structure, and the generated network is simplified applying the "Optimal Brain Surgeon" algorithm proposed by B. Hassibi, D. G. Stork and G.J. Wolf. This procedure reduces neural network plasticity without drastically damaging the learning performance, thus making the descriptive power of produced neural networks similar to the descriptive power of Łukasiewicz logic language and simplifying the translation between symbolic and connectionist structures. We used this method in the reverse engineering problem of finding the formula used on the generation of a given truth table. For real data sets the method is particularly useful for attribute selection, on binary classification problems defined using nominal attributes, where each instance has a level of uncertainty associated with it.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.