Artificial neural networks with an infinite number of nodes

Blekas, Konstantinos; Lagaris, Isaac E.

doi:10.1088/1742-6596/915/1/012006

Cited by 2 publications

(3 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While FWNN employs only ten and 12 parameters for the 1d and 2d datasets, MLP and RBF networks require a significantly larger number in the range [31 − 301] and [41 − 401] , respectively, in order to achieve a comparable test error. For these datasets, a plethora of experiments and related results may be found in [3].…”

Section: Experiments With Simulated Datasetsmentioning

confidence: 99%

“…We observe that all points are scattered symmetrically around and near to the diagonal x = y line that represents the perfect match. For the remaining (Elevators, Kin40k, Pole Telecomm, Pumadyn32-nm) datasets, 3 the FWNN results are listed in Table 9 along with results provided by a state-of-the-art Gaussian process approach reported in [19]. In spite its simplicity, the FWNN's performance is better or similar to that of a sophisticated, high-demanding, state-of-the-art method.…”

Section: Large-scale Experimentsmentioning

confidence: 99%

“…Preliminary results assessing the performance of FWNNs have been reported earlier [3] and have been presented at the Sofianos-2017 international symposium. The substitution of discrete weights by continuous functions has been also considered in [30], where, however, the activation is restricted to be an odd function, and the weights are either piecewise constant or piecewise affine functions.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Functionally weighted neural networks: frugal models with high accuracy

Blekas

Lagaris

2020

SN Appl. Sci.

View full text Add to dashboard Cite

In this article, we introduce the "functionally weighted neural network, " a new addition to the rich collection of artificial neural networks. Instead of a finite number of discrete nodes, we consider an infinite number of continuously distributed nodes. The weights assume a functional form, and the sum over the nodes becomes an integral. The gain is a significant reduction in the number of adjustable parameters, accompanied by an enhanced generalization performance. To quantitatively assess the quality of this new network, we have performed numerical experiments on a number of benchmark datasets. Comparison with state-of-the-art techniques reveals the advantages of the proposed method and emphasizes its modeling potential.

show abstract

Section: Experiments With Simulated Datasetsmentioning

confidence: 99%

Section: Large-scale Experimentsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Functionally weighted neural networks: frugal models with high accuracy

Blekas

Lagaris

2020

SN Appl. Sci.

View full text Add to dashboard Cite

show abstract

Variance Counterbalancing for Stochastic Large-scale Learning

Lagari

Tsoukalas

Lagaris

2020

Int. J. Artif. Intell. Tools

View full text Add to dashboard Cite

Stochastic Gradient Descent (SGD) is perhaps the most frequently used method for large scale training. A common example is training a neural network over a large data set, which amounts to minimizing the corresponding mean squared error (MSE). Since the convergence of SGD is rather slow, acceleration techniques based on the notion of “Mini-Batches” have been developed. All of them however, mimicking SGD, impose diminishing step-sizes as a means to inhibit large variations in the MSE objective. In this article, we introduce random sets of mini-batches instead of individual mini-batches. We employ an objective function that minimizes the average MSE and its variance over these sets, eliminating so the need for the systematic step size reduction. This approach permits the use of state-of-the-art optimization methods, far more efficient than the gradient descent, and yields a significant performance enhancement.

show abstract

Artificial neural networks with an infinite number of nodes

Abstract: Abstract.A new class of Artificial Neural Networks is described incorporating a node density function and functional weights. This network containing an infinite number of nodes, excels in generalizing and possesses a superior extrapolation capability.

Cited by 2 publications

References 21 publications

Functionally weighted neural networks: frugal models with high accuracy

Functionally weighted neural networks: frugal models with high accuracy

Variance Counterbalancing for Stochastic Large-scale Learning

Contact Info

Product

Resources

About