A new strategy for adaptively constructing multilayer feedforward neural networks

Ma, Liying; Khorasani, K.

doi:10.1016/s0925-2312(02)00597-0

Cited by 77 publications

(27 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The number of neurons in input layer is the same as the number of decision variables and there is only one neuron in the output layer. The structure of the network adaptively changes in the algorithm (see Ma and Khorasani 2003 for such methods). We start out the network in the first iteration with one hidden layer that has only two neurons (Figure 4.3).…”

Section: Metamodelingmentioning

confidence: 99%

Continuous optimization via simulation using Golden Region search

Kabirian

Ólafsson

2011

European Journal of Operational Research

View full text Add to dashboard Cite

Section: Metamodelingmentioning

confidence: 99%

Continuous optimization via simulation using Golden Region search

Kabirian

Ólafsson

2011

European Journal of Operational Research

View full text Add to dashboard Cite

“…Those parameters are then optimized with respect to the dataset. During learning phase, the topology of the neural network plays a significant role in whether or not the network can be trained to learn a particular dataset [5,6,7]. A simple topology will result in a network that cannot learn to approximate a complex function, while a large topology is probable to result in a network losing its generalization capability.…”

Section: Introductionmentioning

confidence: 99%

Convergence Rate in a Destructive Neural Network With /Without Thresholds in The Output Layer

Mahmoud

2016

International Journal of Internet Education

View full text Add to dashboard Cite

Neural networks are a practicable solution for the extraction of accurate knowledge, where the data being mined can be so noisy, due to their relatively good tolerance to noisy and generalization ability and the performance of a neural network is directly related to its parameters and architecture. The degree of complexity of ANN increases exponentially as a factor of the numbers of input and hidden nodes. The complexity problem can be improved by constructing the structure of the network based on constructive learning and destructive learning. So, the objective of the network is to learn or to discover some involvement between input and output patterns to find the structure of the input patterns. The learning process is achieved through the modification of the connection weights between units. It is known as the network's topology which determines the network's final behavior. The goal of finding an optimal topology is to minimize an error function while conserving generalization capabilities. In our work, the destructive topology is used: first the algorithm trains a big size of neural network on the data, and then prunes it to increase its generalization capability while preserving its accuracy. The present paper introduces destructive neural network learning techniques and presents the analysis of the convergence rate of the error in a neural network with and without threshold in the output layer. The present paper introduces destructive neural network learning techniques and presents the analysis of the convergence rate of the error in a neural network with and without threshold.

show abstract

“…Experiments showed that there was no difference between the generalization capability of both flat and standard CCN but flat CCN learned more efficiently because there were fewer connections to train. CCN has been widely investigated and inspired the many new proposals in the literature [19][20][21][22][23]. The reviews of constructive algorithms that belong to CCN family are given in [3,24].…”

Section: Introductionmentioning

confidence: 99%

An empirical comparison of cascade and flat variants of incremental node creation algorithm

Sharma

Chandra

2013

CSIT

View full text Add to dashboard Cite

We propose incremental node creation algorithm (INCA). INCA emphasizes on architectural adaptation and functional adaptation in a unified frame work. INCA starts from a single hidden node and then trains node one by one incrementally. Two variants of INCA are developed, namely cascade and flat. In the cascade variant, every hidden node is added in a new hidden layer that is connected to the network inputs and all pre-existing hidden nodes. In contrast, the flat variant adds node one by one to a single hidden layer. Sixteen regression problems are used to investigate which network growing strategy provides the better generalization performance. Simulation results reveal that both architectures perform well on all the investigated regression problems of varying complexities. In general, cascade is better than flat architecture except some real world problems. The trigonometric sine activation function provides better approximation capability than log-sigmoid function except some regression problems.

show abstract

A new strategy for adaptively constructing multilayer feedforward neural networks

Cited by 77 publications

References 24 publications

Continuous optimization via simulation using Golden Region search

Continuous optimization via simulation using Golden Region search

Convergence Rate in a Destructive Neural Network With /Without Thresholds in The Output Layer

An empirical comparison of cascade and flat variants of incremental node creation algorithm

Contact Info

Product

Resources

About