PCA and Gaussian noise in MLP neural network training improve generalization in problems with small and unbalanced data sets

Abstract

Machine learning approaches have been successfully applied for automatic decision support in several domains. The quality of these systems, however, degrades severely in classification problems with small and unbalanced data sets for knowledge acquisition. Inherent to several realworld problems, data sets with these characteristics are the reality to be tackled by learning algorithms, but the small amount of data affects the classifiers' generalization power while the imbalance in class distribution makes the classifiers biased towards the larger classes. Previous work had addressed these data constraints with the addition of Gaussian noise to the input patterns' variables during the iterative training process of a MultiLayer perceptron (MLP) neural network (NN). This paper improves the quality of such classifier by decorrelating the input variables via a Principal Component Analysis (PCA) transformation of the original input space before applying additive Gaussian noise to each transformed variable for each input pattern. PCA transformation prevents the conflicting effect of adding decorrelated noise to correlated variables, an effect which increases with the noise level. Three public data sets from a well-known benchmark (Proben1) were used to validate the proposed approach. Experimental results indicate that the proposed methodology improves the performance of the previous approach being statistically better than the traditional training method (95% confidence) in further experimental set-ups.

5FU-loaded PCL/Chitosan/Fe3O4 Core-Shell Nanofibers Structure: An Approach to Multi-Mode Anticancer System

“…In order to prevent over fitting, sufficient number of training data are essential. Since training data were limited due to experimental restrictions, Gaussian noise exertion on real data 25 and k-fold cross validation method 26 were applied. Finally, Gaussian noise method was selected according to the training error.…”

Optimización y clasificación de señales EMG a través de métodos de reconocimiento de patrones

“…Se ha demostrado que una red MLP con una capa oculta y la función sigmoidal es un aproximador universal de cualquier función continua en un conjunto compacto. Las redes neuronales MLP pueden trabajar como cualquier clasificador no lineal o como una función de regresión [12], [13] y [14].…”

The Classification of Noise-Afflicted Remotely Sensed Data Using Three Machine-Learning Techniques: Effect of Different Levels and Types of Noise on Accuracy

“…BPNNs in remotely sensed image classification applications have been widely reviewed [41][42][43]. They were also reported as very effective to use in noise reduction [44,45] and robust to noises when trained by noise data [46]. These researchers have mutually concluded that the BPNN approach is feasible for the classification of remote sensing imagery.…”