Survey and critique of techniques for extracting rules from trained artificial neural networks

Data plays an important role in our daily life. Thus, data collection, storage, maintenance and processing continue to attract considerable attention. Data may exist in various formats, ranging from unstructured to structured as the two extremes. Traditionally, researchers and practitioners cooperated and developed various data models which form the main foundation for existing database management systems. The relational data model is still dominating despite the rapid development in the techniques used for data collection, storage and processing. Further, a relational database management system supports a structured query language (SQL) for data processing, and it is not possible to access and retrieve data from a relational database without knowing how to use SQL. However, the wide usage of relational databases motivated researchers to develop more user friendly interfaces which would allow a larger population of users to access relational databases. Such interfaces range from visual to natural language based. This thesis contributes a question driven query model which falls under the natural language based category. The target is to make databases reachable by a larger population, especially after the Internet increased database availability. The proposed model supports fuzziness where every user is given the freedom to define his/her own understanding of fuzzy terms. The developed system absorbs the fuzzy understanding of each user to utilize it while deciding on the result to be communicated back as answer to the raised question. Data mining techniques are employed to guide users in defining their fuzzy understanding. The developed model is intended to help users to retrieve the data they want from a relational database without expecting them to know SQL. In the current version only questions written in English are allowed. The system handles different types of questions, such as (1) simple questions, (2) complex questions with inner joins and where conditions, (3) questions that involves the usage of aggregate functions (e.g., min, max, etc.), and (4) questions with fuzzy i terms. The reported test results demonstrate the effectiveness of the developed system in handling various types of questions raised by a heterogeneous set of users ranging from professionals to naive.

show abstract

“…ANN have proven their efficiency in dependency extraction (see Section 2.1.2) which we will use in our study [5,29].…”

Section: Classificationmentioning

confidence: 99%

Integrating flexibility and fuzziness into a question driven query model

Sarhan

Rokne

Alhajj

2018

Information Sciences

View full text Add to dashboard Cite

show abstract

“…A rule extraction algorithm can be roughly divided into two main categories: decompositional and pedagogical (Andrews et al, 1995). Decompositional algorithms take the underlying classification algorithm into account.…”

Section: Rule Extractionmentioning

confidence: 99%

Growing Hierarchical Self-organizing Maps and Statistical Distribution Models for Online Detection of Web Attacks

Zolotukhin

Hämäläinen

Juvonen

2013

Lecture Notes in Business Information Processing

View full text Add to dashboard Cite

“…BSVM improves largely the interpretability of standard SVMs by the use of queries of type (1). In this section we illustrate how our method takes us one step further towards interpretability via the use of the visualization tool To do this, we have considered two databases with different characteristics, namely bupa and sonar, which have respectively 6 and 60 predictor variables.…”

Section: Interpretabilitymentioning

confidence: 99%

“…Since the obtained classification rule is based on queries of type (1), the critical values and intervals of the predictor variables are identified, as done by Classification Trees.…”

Section: Introduction and Literature Reviewmentioning

confidence: 99%

“…Binarizing continuous predictor variables has also been proposed in the so-called rule extraction procedures, within SVM [3,4,16,27,29] and Neural Networks as well, e.g. [1,2,13]. When a rule extraction method is applied to a classifier, one obtains an alternative classifier which hopefully have a similar behavior on data, but is more interpretable, since it is based on simple rules, such as those derived from queries of type (1).…”

Section: Introduction and Literature Reviewmentioning

confidence: 99%

See 1 more Smart Citation

Binarized Support Vector Machines

Carrizosa

Martín-Barragán

Morales

2010

INFORMS Journal on Computing

View full text Add to dashboard Cite

The widely used Support Vector Machine (SVM) method has shown to yield very good results in Supervised Classification problems. Other methods such as Classification Trees have become more popular among practitioners than SVM thanks to their interpretability, which is an important issue in Data Mining.In this work, we propose an SVM-based method that automatically detects the most important predictor variables, and the role they play in the classifier. In particular, the proposed method is able to detect those values and intervals which are critical for the classification. The method involves the optimization of a Linear Programming problem, with a large number of decision variables. The numerical experience reported shows that a rather direct use of the standard Column-Generation strategy leads to a classification method which, in terms of classification ability, is competitive against the standard linear SVM and Classification Trees. Moreover, the proposed method is robust, i.e., it is stable in the presence of outliers and invariant to change of scale or measurement units of the predictor variables.When the complexity of the classifier is an important issue, a wrapper feature selection method is applied, yielding simpler, still competitive, classifiers. In this work, we propose an SVM-based method that automatically detects the most important predictor variables, and the role they play in the classifier. In particular, the proposed method is able to detect those values and intervals which are critical for the classification. The method involves the optimization of a Linear KeywordsProgramming problem with a large number of decision variables. The numerical experience reported shows that a rather direct use of the standard Column-Generation strategy leads to a classification method which, in terms of classification ability, is competitive against the standard linear SVM and Classification Trees. Moreover, the proposed method is robust, i.e., it is stable in the presence of outliers and invariant to change of scale or * This work has been partially supported by projects MTM2005-09362-C03-01 of MEC, Spain, and FQM-329 of Junta de Andalucía, Spain.1 measurement units of the predictor variables.When the complexity of the classifier is an important issue, a wrapper feature selection method is applied, yielding simpler, still competitive, classifiers.

show abstract

Survey and critique of techniques for extracting rules from trained artificial neural networks

Cited by 975 publications

References 12 publications

Integrating flexibility and fuzziness into a question driven query model

Integrating flexibility and fuzziness into a question driven query model

Growing Hierarchical Self-organizing Maps and Statistical Distribution Models for Online Detection of Web Attacks

Binarized Support Vector Machines

Contact Info

Product

Resources

About