Training the random neural network using quasi-Newton methods

Likas, Aristidis; Stafylopatis, Andreas

doi:10.1016/s0377-2217(99)00482-8

Cited by 61 publications

(34 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…If the new weight is still negative, repeat until obtaining a positive number or stop the loop using some control parameter. • An alternative option was presented in [9]. The authors propose a change of variable: instead of using w + i,j and w − i,j they use new variables β + i,j and β − i,j , such that:…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

A more powerful random neural network model in supervised learning applications

Basterrech

Rubino

2013

2013 International Conference on Soft Computing and Pattern Recognition (SoCPaR)

View full text Add to dashboard Cite

Since the early 1990s, Random Neural Networks (RNNs) have gained importance in the Neural Networks and Queueing Networks communities. RNNs are inspired by biological neural networks and they are also an extension of open Jackson's networks in Queueing Theory. In 1993, a learning algorithm of gradient type was introduced in order to use RNNs in supervised learning tasks. This method considers only the weight connections among the neurons as adjustable parameters. All other parameters are deemed fixed during the training process. The RNN model has been successfully utilized in several types of applications such as: supervised learning problems, pattern recognition, optimization, image processing, associative memory.In this contribution we present a modification of the classic model obtained by extending the set of adjustable parameters. The modification increases the potential of the RNN model in supervised learning tasks keeping the same network topology and the same time complexity of the algorithm. We describe the new equations implementing a gradient descent learning technique for the model.

show abstract

Section: Discussionmentioning

confidence: 99%

“…This method is an on-line procedure (this means that each of the K data pairs is read at a time), based on the delta rule [7]. Additionally, Quasi-Newton methods for using RNN to solve supervised learning problems were introduced in [8], [9].…”

Section: A Random Neural Network In Supervised Learning Tasksmentioning

confidence: 99%

A more powerful random neural network model in supervised learning applications

Basterrech

Rubino

2013

2013 International Conference on Soft Computing and Pattern Recognition (SoCPaR)

View full text Add to dashboard Cite

show abstract

“…, N a . In a standard RandNN, the neuron's loads are computed solving the expressions (1), (2) and (3). More precisely, input neurons behave as a M/M/1 queues.…”

Section: A New Reservoir Computing Methodmentioning

confidence: 99%

Echo State Queueing Network: A new reservoir computing learning tool

Basterrech

Rubino

2013

2013 IEEE 10th Consumer Communications and Networking Conference (CCNC)

View full text Add to dashboard Cite

In the last decade, a new computational paradigm was introduced in the field of Machine Learning, under the name of Reservoir Computing (RC). RC models are neural networks which a recurrent part (the reservoir) that does not participate in the learning process, and the rest of the system where no recurrence (no neural circuit) occurs. This approach has grown rapidly due to its success in solving learning tasks and other computational applications. Some success was also observed with another recently proposed neural network designed using Queueing Theory, the Random Neural Network (RandNN). Both approaches have good properties and identified drawbacks.In this paper, we propose a new RC model called Echo State Queueing Network (ESQN), where we use ideas coming from RandNNs for the design of the reservoir. ESQNs consist in ESNs where the reservoir has a new dynamics inspired by recurrent RandNNs. The paper positions ESQNs in the global Machine Learning area, and provides examples of their use and performances. We show on largely used benchmarks that ESQNs are very accurate tools, and we illustrate how they compare with standard ESNs.

show abstract

“…The architecture of a neural network is determined by all the connections in the network and transfer functions of the neurons [41] The backpropagation algorithm proposed by [42] is the most popular algorithm to train ANNs. Moreover, advanced methods like Marquardt [43][44][45], Quasi-Newton [46], or conjugating gradient algorithms [47,48] are also very popular. Due to their application in dynamic environments, these classic learning methods have to be modified to fulfill three important requirements:…”

Section: Artificial Neuralmentioning

confidence: 99%

Modeling of Throughput in Production Lines Using Response Surface Methodology and Artificial Neural Networks

et al. 2018

View full text Add to dashboard Cite

The problem of assigning buffers in a production line to obtain an optimum production rate is a combinatorial problem of type NP-Hard and it is known as Buffer Allocation Problem. It is of great importance for designers of production systems due to the costs involved in terms of space requirements. In this work, the relationship among the number of buffer slots, the number of work stations, and the production rate is studied. Response surface methodology and artificial neural network were used to develop predictive models to find optimal throughput values. 360 production rate values for different number of buffer slots and workstations were used to obtain a fourth-order mathematical model and four hidden layers' artificial neural network. Both models have a good performance in predicting the throughput, although the artificial neural network model shows a better fit ( = 1.0000) against the response surface methodology ( = 0.9996). Moreover, the artificial neural network produces better predictions for data not utilized in the models construction. Finally, this study can be used as a guide to forecast the maximum or near maximum throughput of production lines taking into account the buffer size and the number of machines in the line.

show abstract

Training the random neural network using quasi-Newton methods

Cited by 61 publications

References 18 publications

A more powerful random neural network model in supervised learning applications

A more powerful random neural network model in supervised learning applications

Echo State Queueing Network: A new reservoir computing learning tool

Modeling of Throughput in Production Lines Using Response Surface Methodology and Artificial Neural Networks

Contact Info

Product

Resources

About