A Back Propagation-Type Neural Network Architecture for Solving the Complete n × n Nonlinear Algebraic System of Equations

Abstract

The objective of this research is the presentation of a neural network capable of solving complete nonlinear algebraic systems of n equations with n unknowns. The proposed neural solver uses the classical back propagation algorithm with the identity function as the output function, and supports the feature of the adaptive learning rate for the neurons of the second hidden layer. The paper presents the fundamental theory associated with this approach as well as a set of experimental results that evaluate the performance and accuracy of the proposed method against other methods found in the literature.

Survey of Methods for Solving Systems of Nonlinear Equations, Part II: Optimization Based Approaches

“…The new updating rule dynamically adjusts the step size during training, and rotates after a set number of iterations between using the Manhattan updating rule and the Adam updating rule to efficiently handle problems with sparse and vanishing gradients. Also expanding upon the work of Margaris et al [174], Goulianas et al introduced the General Back-propagation with Adapative Learning Rate (GBALR) algorithm which allows the activation functions of the second hidden layer to be any function, including non-algebraic functions [176].…”