Selective Presentation of Training Set to Back-Propagation Neural Networks

Abstract

We investigate how training patterns should be presented to a back-propagation neural network (BPNN) so as to train the BPNN with a small deviation of training patterns and to improve the BPNN's learning speed. First, we explain the problem with a conventional learning technique, in which all training patterns are presented to a BPNN equally. Then, we propose a selective presentation of training set to a BPNN. In a proposed technique, using several criterion values for both the mean summed squared error and individual summed squared errors, we detect poorly-trained patterns and present them more often. The effectiveness of the proposed technique is confirmed by evaluation experiments using mesh patterns extracted from handwritten digits and two-dimensional Gaussian distribution data.

An efficient learning method for layered neural networks based on selection of training data and input characteristics of an output layer unit

“…(9)] is satisfied, when the catch condition for the target value is satisfied, the decision whether to end or continue learning is determined with respect to the set RMSE value. (9)] is satisfied, when the catch condition for the target value is satisfied, the decision whether to end or continue learning is determined with respect to the set RMSE value.…”

“…(7) Whether or not the condition for reducing amplitude [Eq. (9)] is satisfied, when the catch condition for the target value is satisfied, the decision whether to end or continue learning is determined with respect to the set RMSE value. That is, for values smaller than the set RMSE value, learning is stopped, and otherwise the weighting coefficient is inherited and the final stage of learning continues.…”

“…However, because NN have a simpler structure than recent neural networks, in practice the BP method and its extensions [4,5] are used in a variety of ways as learning rules. Considerable research has been performed on methods of determining the learning coefficient after proceeding efficiently with learning, methods of selecting or of adding training data, and learning methods that use oscillation (oscillation characteristics) [6][7][8][9][10][11][12]. In this paper we propose a method of resolving this problem.…”

“…In this paper we propose a method of resolving this problem. Considerable research has been performed on methods of determining the learning coefficient after proceeding efficiently with learning, methods of selecting or of adding training data, and learning methods that use oscillation (oscillation characteristics) [6][7][8][9][10][11][12]. The use of an inertial coefficient is also assumed to be an important factor for improving learning speed and suppressing oscillation on a learning curve.…”

“…The pairing method [6], which selects pairs of nearest neighbor data in a class, and the ADS method, which uses data for which errors increase when excluded, have been proposed [7] as selection methods for training data. There are also methods [9] that monitor error during learning so as to learn without dividing the training data into classes with large errors.…”