The Upstart Algorithm: A Method for Constructing and Training Feedforward Neural Networks

Frean, Marcus

doi:10.1162/neco.1990.2.2.198

Cited by 437 publications

(185 citation statements)

References 1 publication

Supporting

Mentioning

177

Contrasting

Unclassified

Order By: Relevance

“…There are some efforts for a systematic way to achieve this such as the research on constructive and pruning algorithms, (Burgess, 1994;Frean, 1990;LeCun et al, 1990;Reed, 1993). The former methods initially assume a minimal ANN and insert nodes and links as warranted whilst the latter proceeds in the opposite way, i.e.…”

Section: Evolutionary Annsmentioning

confidence: 99%

“…Up-to-date designing a (near) optimal network architecture is made by a human expert and requires a tedious trial and error process. Especially automatic determination fall into two broad categories: constructive and pruning algorithms (Burgess, 1994;Frean, 1990;LeCun, Denker, & Solla, 1990;Reed, 1993), where many deficiencies and limitations have been reported (Angeline, Sauders, & Pollack, 1994). Efforts have been then focused on evolutionary algorithms (EAs) (Back & Schwefel, 1993) particularly over Genetic Algorithm (GA) (Goldberg, 1989) and Evolutionary Programming (EP), (Fayyad, Shapire, Smyth, & Uthurusamy, 1996) for both training and evolving ANNs.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Evolutionary artificial neural networks by multi-dimensional particle swarm optimization

et al. 2009

View full text Add to dashboard Cite

Section: Evolutionary Annsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Evolutionary artificial neural networks by multi-dimensional particle swarm optimization

et al. 2009

View full text Add to dashboard Cite

“…Moreover, a constructive (add node iteratively) and destructive (prune nodes iteratively) method [196]. However, the constructive and the destructive methods for optimizing architecture are no different from the (Fig.…”

Section: Architecture Plus Weight Optimizationmentioning

confidence: 99%

Metaheuristic design of feedforward neural networks: A review of two decades of research

Ojha

Abraham

Snel

2017

Engineering Applications of Artificial Intelligence

441

154

View full text Add to dashboard Cite

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era.

show abstract

“…Upstart algorithm [3] constructs a binary tree of threshold neurons. First an output layer of M neurons is trained, if patterns are correctly classified, it terminates, else it finds a neuron that makes most number of errors, if it is wrongly-on or wrongly-off, daughter neurons are added to correct errors.…”

Section: Introductionmentioning

confidence: 99%

“…Each neuron is designed to exclude a cluster of patterns belonging to same class. The weights are the inter pattern distances [3] .…”

Section: Introductionmentioning

confidence: 99%

New Constructive Neural Network Architecture for Pattern Classification

Sridhar¹,

Ponnavaikko²

2009

J. of Computer Science

View full text Add to dashboard Cite

Problem statement: Constructive neural network learning algorithms provide optimal ways to determine the architecture of a multi layer perceptron network along with learning algorithms for determining appropriate weights for pattern classification problems. These algorithms initially start with small network and dynamically allow the network to grow by adding and training neurons as needed until a satisfactory solution is found. The constructive neural network training is performed via feed forward paradigm under supervised training considerations. These supervised methods often make the network size grow exponentially, or, the network lacks generalization. To address these problems a new method for learning in constructive neural networks is necessary. Approach: To address these issues a new Multicategory Tiling architecture was chosen for its simple topology and an improved adaptive resonance theory unsupervised training algorithm was used with proper weight setting to train the constructive networks on binary sequence patterns. The results and performance of the new algorithm was compared with existing constructive neural network architectures and tabulated. Results: The new architecture with improved training algorithm offer faster convergence in learning, the nodes required for storage are less and the generalization of pattern classification was achieved in comparison with existing algorithms. Conclusion: Constructive neural networks could be trained using unsupervised algorithm to achieve better performance in comparison with existing supervised algorithms.

show abstract

The Upstart Algorithm: A Method for Constructing and Training Feedforward Neural Networks

Cited by 437 publications

References 1 publication

Evolutionary artificial neural networks by multi-dimensional particle swarm optimization

Evolutionary artificial neural networks by multi-dimensional particle swarm optimization

Metaheuristic design of feedforward neural networks: A review of two decades of research

New Constructive Neural Network Architecture for Pattern Classification

Contact Info

Product

Resources

About