Abstract- Feature subset selection is a common and key\ud problem in many classification and regression tasks. It can\ud be viewed as a multi-objective optimisation problem, since,\ud in the simplest case, it involves feature subset size\ud minimisation and performance maximisation. Here, a\ud multiobjective evolutionary approach is proposed for\ud feature selection. A novel commonality-based crossover\ud operator is introduced and placed in the multiobjective\ud evolutionary setting. This specialised operator helps to\ud preserve building blocks with promising performance. The\ud multiobjective evolutionary algothim employs the novel\ud crossover operator in order to evolve a diverse population of feature subsets with different subset size/performance\ud trade-offs. Selection bias reduction is achieved by means of\ud resampling. We argue that this is a generic approach,\ud which can be used in many modelling problems. It is applied\ud to feature selection on different neural network\ud architectures. Results from experiments with high\ud dimensional benchmarking data sets are given

No abstract

No abstract

Recently there has been a lot of interest in the extraction of symbolic rules from neural networks. The work described in this paper is concerned with an evaluation and comparison of the accuracy and complexity of symbolic rules extracted from radial basis function networks and multi-layer perceptrons. Here we examine the ability of rule extraction algorithms to extract meaningful rules that describe the overall performance of a particular network. In addition, the research also highlights the suitability of a speci c neural network architecture for particular classi cation problems. The research carried out on the extracted rule quality and complexity also has a direct bearing on the use of rule extraction algorithms for data mining and knowledge discovery. IntroductionThe work described in this paper is concerned with an evaluation of the accuracy and complexity of symbolic rules extracted from radial basis function (RBF) networks and multi-layer perceptrons (MLP). RBF neural networks 5] and MLP networks 4] are two of the most widely used neural network architectures. RBF networks are a localist type of learning technique 3]. Local learning systems generally contain elements that are responsive to only a limited section of the input space. This may e n tail separate storage in memory for each pattern unless the representational elements are able to cover (as in the case of RBF hidden units) a given area around the input pattern. This is quite di erent from the distributed approach o f MLP networks. MLP's are able to store many patterns within a limited memory, i.e. the learned patterns are stored across all weights and thresholds. This property i s known as superposition and enables the e cient storage and recall of individual patterns. However, both types of networks are good at pattern recognition and are robust classi ers, with the ability to generalize in making decisions about imprecise input data. They o er robust solutions to a v ariety o f classi cation problems such a s speech, character and signal recognition, as well as functional prediction and system modeling where the physical processes are not understood or are highly complex. The main di erence is that RBF networks may require more hidden units than MLP's to represent the same data set.The local nature of RBF networks makes them a suitable platform for performing rule extraction. Here we examine the ability of rule extraction algorithms to extract meaningful rules that describe the overall performance of a particular network. The research carried out on the extracted rule quality and complexity also has a direct bearing on the use of rule extraction algorithms for data mining and knowledge discovery. Rule extraction is recognized as a powerful technique for neuro-symbolic integration within hybrid systems 15 8].To illustrate how di erent classi ers can partition the data space and thereby produce varying accuracies, gure 1 shows the decision boundaries for a RBF and a MLP network on a two class problem. The MLP uses one or more hyperpla...

No abstract