Levenberg--Marquardt Training Algorithms for Random Neural Networks

Basterrech, Sebastián; Mohammed, Samir Jasim; Rubino, Gerardo; Soliman, Mostafa I.

doi:10.1093/comjnl/bxp101

Cited by 58 publications

(38 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Levenberg-Marquardt, Levenberg [18], Marquardt [19], "LM" is a commonly used training algorithm in neural networks, Hagan and Menhaj [20], Smaoui and Al-Yakoob [21], Bahram et al [22], Basterrech et al [23], that avoids calculating the Hessian matrix and has the goal of minimising a non-linear function. In mathematical terms, the goodness of the estimatedŷ values and the actual y values can be described using a chi distributed error that can be computed using the formula propose by Gavin [24]:…”

Section: Levenberg Marquardtmentioning

confidence: 99%

Bayesian neural networks for the optimisation of biological clocks in humans

Alfonso

González

2020

Preprint

View full text Add to dashboard Cite

DNA methylation is related to aging. Some researchers, such as Horvath or Levine have managed to create epigenetic and biological clocks that predict chronological age using methylation data. These authors used Elastic Net methodology to build age predictors that had a high prediction accuracy. In this article, we propose to improve their performance by incorporating an additional step using neural networks trained with Bayesian learning. We shown that this approach outperforms the results obtained when using Horvath's method, neural networks directly, or when using other training algorithms, such as Levenberg-Marquardt's algorithm. The R-squared value obtained when using our proposed approach in empirical (out-of sample) data was 0.934, compared to 0.914 when using a different training algorithm (Levenberg Marquard), or 0.910 when applying the neural network directly (e.g. without first reducing the dimensionality of the data). The results were also tested in independent datasets that were not used during the training phase. Our method obtained better R-squared values and RMSE than Horvath's and Levine's method in 8 independent datasets. We demonstrate that building an age predictor using a Bayesian based algorithm provides accurate age predictions. This method is implemented in an R function, which is available through a package created for predicting purposes and is applicable to methylation data. This will help to elucidate the role of DNA methylation age in complex diseases or traits related to aging.

show abstract

Section: Levenberg Marquardtmentioning

confidence: 99%

Bayesian neural networks for the optimisation of biological clocks in humans

Alfonso

González

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…The remaining parameters have been chosen at random with T ij = U (0, 1) and c i = U int (1,4 the number of injured at location I j is also chosen from the uniform distribution such that…”

Section: Training Architecturementioning

confidence: 99%

Fast Non-Negative Least-Squares Learning in the Random Neural Network

Timotheou

2016

Prob. Eng. Inf. Sci.

View full text Add to dashboard Cite

The random neural network is a biologically inspired neural model where neurons interact by probabilistically exchanging positive and negative unit-amplitude signals that has superior learning capabilities compared to other artificial neural networks. This paper considers non-negative least squares supervised learning in this context, and develops an approach that achieves fast execution and excellent learning capacity. This speedup is a result of significant enhancements in the solution of the non-negative least-squares problem which regard (a) the development of analytical expressions for the evaluation of the gradient and objective functions and (b) a novel limited-memory quasi-Newton solution algorithm. Simulation results in the context of optimizing the performance of a disaster management problem using supervised learning verify the efficiency of the approach, achieving two orders of magnitude execution speedup and improved solution quality compared to state-of-the-art algorithms.

show abstract

“…The Levenberg-Marquardt backpropagation algorithm was proved as an efficient method to update weights and biases of neural network (Hagan et al 1994, Ampazis et al 2000. The details of the Levenberg-Marquardt backpropagation algorithm can be found in a number of references (Hagan et al 1994, Haykin 2008, Basterrech et al 2011, and it is applied in this study.…”

Section: Artificial Neural Networkmentioning

confidence: 99%

Practical Implementation of Knowledge-Based Approaches for SAGD Production Analysis

Leung

Zanon

et al. 2014

Day 1 Tue, June 10, 2014

View full text Add to dashboard Cite

Quantitative appraisal of different operating areas and assessment of uncertainty due to reservoir heterogeneities are crucial elements in optimization of production and development strategies in oil sands operations. Although detailed compositional simulators are available for recovery performance evaluation for SAGD, the simulation process is usually deterministic and computationally demanding, and it not quite practical for real-time decision-making and forecasting. Data mining and machine learning algorithms provide efficient modeling alternatives, particularly when the underlying physical relationships between system variables are highly complex, non-linear, and possibly uncertain.In this study, a comprehensive training set encompassing SAGD field data compiled from numerous publicly-available sources is studied. Exploratory data analysis is carried out to interpret and extract relevant attributes describing characteristics associated with reservoir heterogeneities and operating constraints. Because of their ease of implementation and computational efficiency, knowledge-based techniques including artificial neural networks (ANN) are employed to facilitate SAGD production performance prediction. Predicting (input) variables including porosity, net-to-gross ratio, saturation, gross pay, normalized shale barrier thickness and distance to well pair, and initial production rate are formulated. Measures such as cumulative production over discrete time intervals are considered as prediction (output) variables. Data records that are comprised of both input and output variables are assembled; the network is trained using the data set to identify all significant patterns and relationships that exist between the input and the output variables. The model is subsequently validated using a cross-verification procedure, during which records that have been excluded at the training stage are presented to the model. This paper demonstrates that knowledge-based techniques can be implemented in a practical manner to analyze large amount of competitor data efficiently. The approach can be integrated directly into most existing reservoir management routines. It can also be readily updated when new information has become available. Given that robust reservoir management and real-time decision-making are major challenges faced by the industry, the data-driven models presented in this paper has great potential to be applied in other recovery projects such as solvent-aided steam injection.

show abstract

Levenberg--Marquardt Training Algorithms for Random Neural Networks

Cited by 58 publications

References 0 publications

Bayesian neural networks for the optimisation of biological clocks in humans

Bayesian neural networks for the optimisation of biological clocks in humans

Fast Non-Negative Least-Squares Learning in the Random Neural Network

Practical Implementation of Knowledge-Based Approaches for SAGD Production Analysis

Contact Info

Product

Resources

About