A Convergence Theorem for Sequential Learning in Two-Layer Perceptrons

Marchand, Mario; Golea, Mostefa; Ruján, P.

doi:10.1209/0295-5075/11/6/001

Cited by 114 publications

(64 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sirat and Nadal proposed a similar algorithm in (Sirat & Nadal, 1990). Other interesting constructive techniques are sequential learning (Marchand, Golea, & Rújan, 1990), the patch algorithm (Barkema, Andree, & Taal, 1993), the oilspot algorithm (Frattale-Mascioli & Martinelli, 1995) and the techniques presented in (Muselli, 1995;Rújan & Marchand, 1989). Mezard and Nadal in (Mezard & Nadal, 1989), proposed a tiling algorithm which starts by training a single unit on the whole training set.…”

Section: Discussionmentioning

confidence: 99%

The constraint based decomposition (CBD) training architecture

Drăghici

2001

Neural Networks

View full text Add to dashboard Cite

The Constraint Based Decomposition (CBD) is a constructive neural network technique that builds a three or four layer neural network and is guaranteed to find a solution for any classification problem. CBD is shown to be able to solve complicated problems in a simple, fast and reliable manner. The technique is further enhanced by two modifications (locking detection and redundancy elimination) which address the training speed and the efficiency of the internal representation built by the network. The redundancy elimination aims at building more compact architectures while the locking detection aims at improving the training speed. The computational cost of the redundancy elimination is negligible and this enhancement can be used for any problem. However, the computational cost of the locking detection is exponential in the number of dimensions and should only be used in low dimensional spaces. The experimental results show the performance of the algorithm presented in a series of classical benchmark problems including the 2-spiral problem and the Iris, Wine, Glass, Lenses, Ionosphere, Lung cancer, Pima indians, Bupa, Tic-Tac-Toe, Balance and Zoo data sets from the CMU machine learning repository. CBD's generalization accuracy is compared with that of C4.5, C4.5 with rules, incremental decision trees, oblique classifiers, linear machine decision trees, CN2, learning vector quantization (LVQ), backpropagation, nearest neighbor, Q* and radial basis functions (RBFs). CBD provides the second best average accuracy on the problems tested as well as the best reliability (the lowest standard deviation).

show abstract

Section: Discussionmentioning

confidence: 99%

The constraint based decomposition (CBD) training architecture

Drăghici

2001

Neural Networks

View full text Add to dashboard Cite

show abstract

“…when it is less than about 75%, the PLR leads to the best or the second best generalization performance. However, there is no possibility of direct comparison of the proposed method to the previous works in terms of the generalization performance, since the most similar methods, those also employing the same data sets [4,5,22,23], do not declare their generalization performances. The generalization performance of the proposed cascade network is discussed in Section 4.5.…”

Section: Algorithmmentioning

confidence: 99%

“…It was proven in [4] that all samples can correctly be classified by the SLA in a 2-layer structure, where the outputs of the first layer become linearly separable and the second layer consists of just 1 neuron. However, the work in [4] dealt with Boolean inputs only. Another algorithm, based on SLA, is the constructive algorithm for real-valued examples (CARVE) [5], which extends the SLA from * Correspondence: ibrahim.genc@medeniyet.edu.tr Boolean inputs to real-valued input cases; it uses a convex hull method for the determination of the network weights instead of the PLR.…”

Section: Introductionmentioning

confidence: 99%

A convergent algorithm for a cascade network of multiplexed dual outputdiscrete perceptrons for linearly nonseparable classification

Genc¹,

Güzeliş²

2014

Turk J Elec Eng & Comp Sci

View full text Add to dashboard Cite

Abstract:In this paper a new discrete perceptron model is introduced. The model forms a cascade structure and it is capable of realizing an arbitrary classification task designed by a constructive learning algorithm. The main idea is to copy a discrete perceptron neuron's output to have a complementary dual output for the neuron, and then to select, by using a multiplexer, the true output, which might be 0 or 1 depending on the given input. Hence, the problem of realization of the desired output is transformed into the realization of the selector signal of the multiplexer. In the next step, the selector signal is taken as the desired output signal for the remaining part of the network. The repeated applications of the procedure render the problem into a linearly separable one and eliminate the necessity of using the selector signal in the last step of the algorithm. The proposed modification to the discrete perceptron brings universality with the expense of getting just a slight modification in hardware implementation.

show abstract

“…Sequential algorithm [7], instead of training neurons to classify a maximal subset of patterns, it trains neurons to sequentially exclude patterns belonging to one class from other. When all patterns are excluded, the internal representation of patterns in hidden layer is linearly separable.…”

Section: A Constructive Neural Network Algorithmsmentioning

confidence: 99%

Improved Adaptive Learning Algorithm forConstructive Neural Networks

Sridhar¹,

Ponnavaikko²

2011

IJCEE

View full text Add to dashboard Cite

Abstract-Constructive Neural Network learning algorithms provide incremental ways to determine the near-minimal architecture of a multi layer perceptron network along with learning algorithms for determining appropriate weights for pattern classification problems. An improved version of adaptive learning algorithm in a structured multilayer networks is proposed in this research work. A proper weight setting for the constructive architecture to solve pattern classification problems is analyzed and tabulated.

show abstract

A Convergence Theorem for Sequential Learning in Two-Layer Perceptrons

Cited by 114 publications

References 9 publications

The constraint based decomposition (CBD) training architecture

The constraint based decomposition (CBD) training architecture

A convergent algorithm for a cascade network of multiplexed dual outputdiscrete perceptrons for linearly nonseparable classification

Improved Adaptive Learning Algorithm forConstructive Neural Networks

Contact Info

Product

Resources

About