Statistical analysis of the parameters of a neuro-genetic algorithm

Abstract

Abstract-Interest in hybrid methods that combine artificial neural networks (ANNs) and evolutionary algorithms (EAs) has grown in the last few years, due to their robustness and ability to design networks by setting initial weight values, by searching the architecture and the learning rule and parameters. However, papers describing the way genetic operators are tested to determine their effectiveness are scarce; moreover, few researchers publish the most suitable values of these operator parameters to solve a given problem. This paper presents an exhaustive analysis of the G-Prop method, and the different parameters the method requires (population size, selection rate, initial weight range, number of training epochs, etc.) are determined. The paper also the discusses the influence of the application of genetic operators on the precision (classification ability or error) and network size in classification problems. When making a detailed statistical analysis of the influence of each parameter, the designer should pay most attention to the parameter presenting values that are statistically most significant. The significance and relative importance of the parameters with respect to the results obtained, as well as suitable values for each, were obtained using ANalysis Of the VAriance (ANOVA). Experiments show the significance of parameters concerning the neural network and learning in the hybrid methods. Combining evolutionary algorithms and neural network learning methods can lead to better results than using those methods alone. Moreover, making the network initial weights evolve is an important factor in the process. The parameters found using this method were used to compare the G-Prop method both to itself with other parameter settings, and to other published methods.Index Terms-ANalysis Of the VAriance (ANOVA), artificial neural networks (ANNs), evolutionary algorithms (EAs), hybrid methods, optimization, statistical analysis.

Using evolution to improve neural network learning: pitfalls and solutions

“…Earlier studies have already shown that evolution can be used to create high performance neural network systems [2][3][4][5][6][7][8][9][10][11][12][13][15][16][17], and this paper has confirmed that even a less than optimal choice of evolutionary approach can still result in the emergence of good individual networks. However, it is now clear from this study that to generate the best possible neural networks, it is important to choose the right evolutionary approach for each particular application.…”

“…There appear not to have been any general investigations into which evolutionary approaches are best to use for this, nor any checks for potentially deleterious side-effects of the evolution, though comparative studies of other factors do exist (e.g. [3,8,9,10,11]). This paper attempts to fill this gap, with particular emphasis on the potential pitfalls of the most obvious evolutionary approaches, and their associated solutions.…”

Classification of surface EMG signals using optimal wavelet packet method based on Davies-Bouldin criterion

“…In order to investigate the performance of the proposed method, the one-way analysis of variance (ANOVA) [5,10] was used. This statistical tool, based on the analysis of the mean variance, is widely used.…”

Evolutionary Fuzzy ARTMAP Neural Networks and their Applications to Fault Detection and Diagnosis

“…In structural evolution of an ANN, the network parameters such as the number of hidden layers/nodes, learning rates, momentum, are optimized (e.g., [18][19][20]). Occasionally, EAs are used for structural evolution whereas the ANN is responsible for weight adaptation through a suitable ANN training algorithm.…”