On Training Efficiency and Computational Costs of a Feed Forward Neural Network: A Review

Laudani, Antonino; Lozito, Gabriele Maria; Fulginei, Francesco Riganti; Salvini, Alessandro

doi:10.1155/2015/818243

Cited by 68 publications

(35 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Filliatre and Racca [20] have studied the PP for speech synthesis. Many such works have been presented in [21][22][23][24][25]. Hu et al [21] have used two distributions, namely Cauchy and laplace and one error function, called Gaussian to generate novel activation functions.…”

Section: Related Workmentioning

confidence: 99%

An efficient learning algorithm for periodic perceptron to test XOR function and parity problem

et al. 2020

View full text Add to dashboard Cite

Artificial neural network (ANN) is an important tool, which is used in numerous fields, such as computer vision, pattern recognition, signal processing, solving optimization problems, and voice analysis and synthesis. Many real-life problems, where the future events play a vital role, are based on the past history. For example, predicting the behavior of stock market indices and electrical load forecasting. In this paper, we establish an efficient learning algorithm for periodic perceptron (PP) in order to test in realistic problems, such as the XOR function and the parity problem. Here, the periodic threshold output function guarantees the convergence of the learning algorithm for the multilayer perceptron. By using the binary Boolean function and the PP in single and multilayer perceptron, XOR problem is solved. The performance of PP is compared with multilayer perceptron and the result shows superiority of PP over the multilayer perceptron.

show abstract

Section: Related Workmentioning

confidence: 99%

An efficient learning algorithm for periodic perceptron to test XOR function and parity problem

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Neural networks usually utilise the same, non-linear xed activation function for all neurons, but the activation function has a signi cant in uence on the learning performance, topology and tness [4,11,13]. Mayer and Schwaiger evolve the activation function of generalized multi-layer perceptrons [16] within the netGEN framework, which uses a genetic algorithm to evolve neural network topologies [9].…”

Section: Evolving Activation Functionsmentioning

confidence: 99%

Evolving parsimonious networks by mixing activation functions

Hagg

Mensing

Asteroth

2017

Proceedings of the Genetic and Evolutionary Computation Conference

View full text Add to dashboard Cite

Neuroevolution methods evolve the weights of a neural network, and in some cases the topology, but li le work has been done to analyze the e ect of evolving the activation functions of individual nodes on network size, which is important when training networks with a small number of samples. In this work we extend the neuroevolution algorithm NEAT to evolve the activation function of neurons in addition to the topology and weights of the network. e size and performance of networks produced using NEAT with uniform activation in all nodes, or homogenous networks, is compared to networks which contain a mixture of activation functions, or heterogenous networks. For a number of regression and classi cation benchmarks it is shown that, (1) qualitatively di erent activation functions lead to di erent results in homogeneous networks, (2) the heterogeneous version of NEAT is able to select well performing activation functions, (3) producing heterogeneous networks that are signi cantly smaller than homogeneous networks.

show abstract

“…[65] Proposes a new MBP centered discriminative feature learning approach that specifically aims at learning a low dimensional feature representation to maximize the global margin of the data and samples from the same class as close as possible. [66] Also recently proposes a performance evaluation of loss functions for speech recognition to enhance the generality of acoustic model by implementing an MBP A more recent work also looked at training efficiency and computational cost, both in general-purpose and in embedded computing environments and a strategy to convert a network configuration between different activation functions without altering the network mapping capabilities in FFNN were also presented in [67]. These literatures serve as a stimulant for this paper.…”

Section: Figure 7 Topological Structure Of Dpfnnmentioning

confidence: 99%

Taxonomy and a Theoretical Model for Feedforward Neural Networks

Benuwa¹,

Zhan²,

Ghansah³

et al. 2017

IJCA

View full text Add to dashboard Cite

Feedforward Neural Network (FFNN) is a surrogate of Artificial Neural Network (ANN) in which links amongst the units do not form a directed cycle. ANNs, akin to the vast network of neurons in the brain (human central nervous system) are usually presented as systems of interweaving connected "neurons" which exchange messages between each other. These connections have numeric hefts that can be adjusted and grounded on experience, enforcing adaptively on neural networks to inputs and learning capabilities. This paper presents a comprehensive review of FFNN with emphasis on implantation issues, which have been addressed by previous approaches. We also propose a theoretical model that exhibits potential superior performances in terms of convergence speed, efficient and effective computation and generality than state of the art models.

show abstract

On Training Efficiency and Computational Costs of a Feed Forward Neural Network: A Review

Cited by 68 publications

References 61 publications

An efficient learning algorithm for periodic perceptron to test XOR function and parity problem

An efficient learning algorithm for periodic perceptron to test XOR function and parity problem

Evolving parsimonious networks by mixing activation functions

Taxonomy and a Theoretical Model for Feedforward Neural Networks

Contact Info

Product

Resources

About