The number of precision bits for operations and data are limited in the hardware implementations of backpropagation (BP). Reduction of rounding error due to this limited precision is crucial in the implementation. The new learning algorithm is based on overestimation of significant error in order to alleviate underflow and omission of weight updating for correctly recognized patterns. While the conventional BP algorithm minimizes the squared error between output signals and supervising data, the new learning algorithm minimizes the weighted error function. In the learning simulation of multifont capital recognition, this algorithm converged recognition accuracy to 100% with only 8-b precision. In addition, the recognition accuracy for characters that did not appear in the training data reached 94.9%. This performance is equivalent to that of a conventional BP with 12-b precision. Moreover, it is found that the performance of the weighted error function is high even when only a small number of hidden neurons is used. Consequently, the algorithm reduces the required amount of weight memory.
Whereas a character segmentation is an essential pre-process for performing a character recognition, this has been an extremely complicated task for Japanese document recognition. The difficulties of it are due to the irregularities of sizes and disposition of Japanese characters in addition to an existence of separated characters.Thus, we have developed a new segmentation method with a Hopfield type neural networks and applied it to handwritten Japanese character strings. A general constraining conditions for segmentation of Japanese characters is expressed as energy functions in the networks and the networks can perform segmentation of Japanese character smngs pliably. Our experimental result showed a probability of correct segmentation of 82.8% in contrast to 75.9% obtained by the conventional method.
SUMMARYThis paper describes the implementation of nonlinear optimization methods into the learning of neural networks (NN) and the speed efficiency of four proposed improvements into the backpropagation algorithm. The problems of the backpropagation learning method are pointed out first, and the efficiency of implementing a nonlinear optimization method as a solution to this problems is described.Two nonlinear optimization methods are selected after inspecting several nonlinear methods from the viewpoint of NN learning to avoid the problem of the backpropagation algorithm. These are the linear search method by Davies, Swann, and Campy (DSC), and the conjugate gradient method by Fletcher and Reeves.The NN learning algorithms with these standard methods being implemented are formulated. Moreover, the following four improvements of the nonlinear optimization methods are proposed to shorten the NN learning time: (a) fast forward calculation in linear search by consuming a larger amount of memories; (b) avoiding the trap to local minimum point in an early stage of linear search; (c) applying a linear search method suitable for parallel processing; and (d) switching the gradient direction using the conjugate gradient method. The evaluation results have shown that all methods described here are effective in shortening the learning time.
We newly developed a handwritten numeric character recognition system with Neural networks designed on Approximate Reasoning Architecture (NARA) and obtained a correct answer rate of 95.41%, an error rate of 0.20% and a rejection rate of 4.38% of handwritten character images.The handwritten character recognition is one of the most difficult target in an area of pattern recognition because of tremendous variation of handwritten images even in a same category of character. NARA which consists of a classifier of input data, several sub-neural networks and an integrator of the outputs of sub-neural networks can realize a stable recognition of tremendous variation of handwritten character images.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.