Determination of Over-Learning and Over-Fitting Problem in Back Propagation Neurl Network

Abstract

A drawback of the error-back propagation algorithm for a multilayer feed forward neural network is over learning or over fitting. We have discussed this problem, and obtained necessary and sufficient Experiment and conditions for over-learning problem to arise. Using those conditions and the concept of a reproducing, this paper proposes methods for choosing training set which is used to prevent over-learning. For a classifier, besides classification capability, its size is another fundamental aspect. In pursuit of high performance, many classifiers do not take into consideration their sizes and contain numerous both essential and insignificant rules. This, however, may bring adverse situation to classifier, for its efficiency will been put down greatly by redundant rules. Hence, it is necessary to eliminate those unwanted rules. We have discussed various experiments with and without over learning or over fitting problem.

Identification of the fuel-thrust dynamics of a gas turbo engine

“…The first simple observation confirms that the steady state gain is nonlinear and depends on the fuel-flow operating range. The second more complicated observation is related to the over-learning phenomenon which is commonly found on auto-constructing identification methods such as auto-constructing neural networks [10,11].…”

Artificial Neural Networks in Cardiology: Analysis of Numerical and Text Data

“…Как известно, для каждой задачи требуется разное количество данных для обучения ИНС. Заранее затруднительно определить, какой объем данных для обучения необходим -недостаточное количество приведет к неточности модели, а излишнее обучение, или переобучение, заставит ИНС не обучаться, а зазубривать данные, что приведет к шуму, который влияет на точность результата [21]. Тем не менее, существуют методы статистической эвристики, которые позволяют определить подходящий размер выборки -данные методы выглядят как специальные коэффициенты масштабирования [22], например, увеличение выборки на определенную константу либо определенный процент для достижения желаемого уровня достоверности.…”

An Adaptive Ensemble Approach to Ambient Intelligence Assisted People Search

“…(2) A binary classifier is usually used as the face recognizer, which is trained to distinguish between two photos belonging to the same person. Because only one or a limited number of photos per person are available to train the recognizer, and due to the over-fitting problem [13] in machine learning, a binary classifier usually achieves a better result than a multiclass classifier. Consequently, heavily unbalanced results ensue, i.e., much more false results occur on positive samples than on negative ones [14].…”

“…When we give a robot a front face photo to learn the features of a new person, we are looking for a mixture matrix A, as in Equation (13), to combine the pre-trained classifiers to obtain a higher recognition rate. Equation 14is a combination of M classifiers, where (•) means the Hadamard product (entrywise product):…”

“…To combine the multiple decision functions, we need to properly estimate the mixture matrix in Equation (13). According to the Bayesian model, we can combine classifiers by their posterior beliefs, as in Equation 15:…”