Implementation of Kolmogorov Learning Algorithm for Feedforward Neural Networks

Abstract

Abstract. We present a learning algorithm for feedforward neural networks that is based on Kolmogorov theorem concerning composition of ndimensional continuous function from one-dimensional continuous functions. A thorough analysis of the algorithm time complexity is presented together with serial and parallel implementation examples.

A three layer neural network can represent any multivariate function

“…Note that the activation functions g k in the output layer depend on f and have to be determined by learning procedures. Some practically useful learning algorithms for such networks were discussed in [19]. By using cubic spline technique of approximation, both for activation and internal transfer functions in Kolmogorov type networks, more efficient approximation of multivariate functions was achieved (see [6]).…”

Variants of Learning Algorithm Based on Kolmogorov Theorem

“…[5]). Functions θ and previous iteration error function e r−1 are then used to compile the outer functions φ r q for q =0,...,m…”

“…The following theorem 3 summarizes our estimation of the total amount of computational time for one iteration (cf. [5]). …”

“…In [5] we have proposed a serial implementation of the Kolmogorov algorithm. Both the theoretical results and practical experiments have shown that the time complexity is relatively big.…”