Constructive algorithm for fully connected cascade feedforward neural networks

Qiao, Junfei; Li, Fanjun; Han, Honggui; Li, Wenjing

doi:10.1016/j.neucom.2015.12.003

Cited by 47 publications

(14 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Yapay sinir ağlarında optimal nöron sayısını belirlemek için kullanılan iki farklı yaklaşım vardır. İlk yaklaşım minimal bir mimari ile başlayıp, nöron ekleyerek ağın kapasitesini geliştirip optimal nöron sayısını elde etmektir [15][16][17][18][19][20]. İkinci yaklaşım ise gereğinden fazla nöron bulunan bir mimari ile başlayıp, gereksiz nöronları budayarak optimum nöron sayısını elde etmektir [13,[21][22][23][24][25][26][27][28][29][30].…”

Section: 𝑖=1unclassified

Investigation of the Optimal Number of Neuron in One-Hidden-Layer Artificial Neural Networks

ÜNAL>

Çiftçi

URGAN

2022

Süleyman Demirel Üniversitesi Fen Edebiyat Fakültesi Fen Dergisi

View full text Add to dashboard Cite

Bu makalede, bir gizli katmanlı yapay sinir ağları için optimal nöron sayısı araştırılmıştır. Bunun için teorik ve istatiksel çalışmalar yapılmıştır. Optimal nöron sayısını bulmak için global minimum bulmak gereklidir. Ancak yapay sinir ağlarının eğitimi konveks olmayan bir problem olduğundan optimizasyon algoritmaları ile global minimum bulmak zordur. Bu çalışmada global minimumu dolayısıyla optimum nöron sayısını bulmak için baskı maliyet fonksiyonu önerilmiştir. Baskı maliyet fonksiyonu yardımıyla global minimumu veren yapay sinir ağı modelinin nöron sayısının, optimal nöron sayısını verdiği gösterilmiştir. Ayrıca baskı maliyet fonksiyonu XOR veri kümesi ve daire veri kümesi üzerinde test edilmiş ve XOR veri kümesi üzerinde %99, daire veri kümesi üzerinde ise %97 başarı elde edilmiştir. Bu veri kümeleri için optimal nöron sayısı tespit edilmiştir.

show abstract

Section: 𝑖=1unclassified

Investigation of the Optimal Number of Neuron in One-Hidden-Layer Artificial Neural Networks

ÜNAL>

Çiftçi

URGAN

2022

Süleyman Demirel Üniversitesi Fen Edebiyat Fakültesi Fen Dergisi

View full text Add to dashboard Cite

show abstract

“…In addition, the Faster Cascade Neural Network (FCNN) was proposed to improve the generalization performance of CCN and improve the existing OLSCN drawbacks: First, the linear dependence of candidate units with the existing u can cause mistake in the new objective function in OLSCN. Second, the w icw modification by the modified Newton Method is based on a second-order H which may result in a local minimum and slow convergence due to heavy computation (Qiao et al , 2016). In FCNN, hidden nodes are generated randomly and remain unchanged as inspired by randomly mapping algorithms.…”

Section: Optimization Techniquesmentioning

confidence: 99%

Machine learning facilitated business intelligence (Part II)

Khan

Chung

Awan

et al. 2019

IMDS

View full text Add to dashboard Cite

Purpose The purpose of this paper is three-fold: to review the categories explaining mainly optimization algorithms (techniques) in that needed to improve the generalization performance and learning speed of the Feedforward Neural Network (FNN); to discover the change in research trends by analyzing all six categories (i.e. gradient learning algorithms for network training, gradient free learning algorithms, optimization algorithms for learning rate, bias and variance (underfitting and overfitting) minimization algorithms, constructive topology neural networks, metaheuristic search algorithms) collectively; and recommend new research directions for researchers and facilitate users to understand algorithms real-world applications in solving complex management, engineering and health sciences problems. Design/methodology/approach The FNN has gained much attention from researchers to make a more informed decision in the last few decades. The literature survey is focused on the learning algorithms and the optimization techniques proposed in the last three decades. This paper (Part II) is an extension of Part I. For the sake of simplicity, the paper entitled “Machine learning facilitated business intelligence (Part I): Neural networks learning algorithms and applications” is referred to as Part I. To make the study consistent with Part I, the approach and survey methodology in this paper are kept similar to those in Part I. Findings Combining the work performed in Part I, the authors studied a total of 80 articles through popular keywords searching. The FNN learning algorithms and optimization techniques identified in the selected literature are classified into six categories based on their problem identification, mathematical model, technical reasoning and proposed solution. Previously, in Part I, the two categories focusing on the learning algorithms (i.e. gradient learning algorithms for network training, gradient free learning algorithms) are reviewed with their real-world applications in management, engineering, and health sciences. Therefore, in the current paper, Part II, the remaining four categories, exploring optimization techniques (i.e. optimization algorithms for learning rate, bias and variance (underfitting and overfitting) minimization algorithms, constructive topology neural networks, metaheuristic search algorithms) are studied in detail. The algorithm explanation is made enriched by discussing their technical merits, limitations, and applications in their respective categories. Finally, the authors recommend future new research directions which can contribute to strengthening the literature. Research limitations/implications The FNN contributions are rapidly increasing because of its ability to make reliably informed decisions. Like learning algorithms, reviewed in Part I, the focus is to enrich the comprehensive study by reviewing remaining categories focusing on the optimization techniques. However, future efforts may be needed to incorporate other algorithms into identified six categories or suggest new category to continuously monitor the shift in the research trends. Practical implications The authors studied the shift in research trend for three decades by collectively analyzing the learning algorithms and optimization techniques with their applications. This may help researchers to identify future research gaps to improve the generalization performance and learning speed, and user to understand the applications areas of the FNN. For instance, research contribution in FNN in the last three decades has changed from complex gradient-based algorithms to gradient free algorithms, trial and error hidden units fixed topology approach to cascade topology, hyperparameters initial guess to analytically calculation and converging algorithms at a global minimum rather than the local minimum. Originality/value The existing literature surveys include comparative study of the algorithms, identifying algorithms application areas and focusing on specific techniques in that it may not be able to identify algorithms categories, a shift in research trends over time, application area frequently analyzed, common research gaps and collective future directions. Part I and II attempts to overcome the existing literature surveys limitations by classifying articles into six categories covering a wide range of algorithm proposed to improve the FNN generalization performance and convergence rate. The classification of algorithms into six categories helps to analyze the shift in research trend which makes the classification scheme significant and innovative.

show abstract

“…However, the random weights sometimes make the column vectors of hidden layer output matrix of the network not full rank [37], i.e. 0 L …”

Section: Contribution Degree Of Unitsmentioning

confidence: 99%

A self-organizing cascade neural network with random weights for nonlinear system modeling

Qiao

Han

et al. 2016

Applied Soft Computing

Self Cite

View full text Add to dashboard Cite

In this paper, a self-organizing cascade neural network (SCNN) with random weights is proposed for nonlinear system modeling. This SCNN is constructed via simultaneous structure and parameter learning processes. In structure learning, the units, which lead to the maximal error reduction of the network, are selected from the candidates and added to the existing network one by one. A stopping criterion based on the training and validation errors is introduced to select the optimal network size to match with a given application. In parameter learning, the weights connected with the output units are incrementally updated without gradients or generalized inverses, while the other weights are randomly assigned and no need to be tuned. Then, the convergence of SCNN is analyzed. Finally, the proposed SCNN is tested on two benchmark nonlinear systems and an actual municipal sewage treatment system. The experiment results show that the proposed SCNN has better performance on nonlinear system modeling than other similar methods.

show abstract

Constructive algorithm for fully connected cascade feedforward neural networks

Cited by 47 publications

References 28 publications

Investigation of the Optimal Number of Neuron in One-Hidden-Layer Artificial Neural Networks

Investigation of the Optimal Number of Neuron in One-Hidden-Layer Artificial Neural Networks

Machine learning facilitated business intelligence (Part II)

A self-organizing cascade neural network with random weights for nonlinear system modeling

Contact Info

Product

Resources

About