Artificial Higher Order Neural Network Training on Limited Precision Processors

Abstract

Previous research suggested Artificial Neural Network (ANN) operation in a limited precision environment was particularly sensitive to the precision and could not take place below a certain threshold level of precision. This study investigates by simulation the training of networks using Back Propagation (BP) and Levenberg-Marquardt algorithms in limited precision to achieve high overall calculation accuracy, using on-line training, a new type of Higher Order Neural Network (HONN) known as the Correlation HONN (CHONN), discrete XOR and continuous optical waveguide sidewall roughness datasets to find the precision at which the training and operation is feasible. The BP algorithm converged to a precision beyond which the performance did not improve. The results support previous findings in literature for ANN operation that discrete datasets require lower precision than continuous datasets. The importance of our findings is that they demonstrate the feasibility of on-line, real-time, low-latency training on limited precision electronic hardware such as Digital Signal Processors (DSPs) and Field Programmable Gate Arrays (FPGAs) to achieve high overall operational accuracy.