Convergence Properties of the Inexact Levenberg-Marquardt Method under Local Error Bound Conditions

Dan, Hiroshige; Yamashita, Nobuo; Fukushima, Masao

doi:10.1080/1055678021000049345

Cited by 93 publications

(61 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, ∇F (x * ) might be singular but nevertheless F (x) may provide a local error bound at x * . One can refer to the examples provided in [8] and [27]. It is wellknown that the Levenberg-Marguardt method has local quadratic convergence when the Jacobian at x * is nonsingular.…”

mentioning

confidence: 99%

A Derivative-Free Algorithm for Least-Squares Minimization

Zhang¹,

Conn²,

Scheinberg³

2010

SIAM J. Optim.

100

103

View full text Add to dashboard Cite

Abstract. We develop a framework for a class of derivative-free algorithms for the least-squares minimization problem. These algorithms are based on polynomial interpolation models and are designed to take advantages of the problem structure. Under suitable conditions, we establish the global convergence and local quadratic convergence properties of these algorithms. Promising numerical results indicate the algorithm is efficient and robust when finding both low accuracy and high accuracy solutions. Comparisons are made with standard derivative-free software packages that do not exploit the special structure of the least-squares problem or that use finite differences to approximate the gradients.

show abstract

mentioning

confidence: 99%

A Derivative-Free Algorithm for Least-Squares Minimization

Zhang¹,

Conn²,

Scheinberg³

2010

SIAM J. Optim.

100

103

View full text Add to dashboard Cite

show abstract

“…We call a solution x * of problem (1.1) a singular solution if G(x * ) is singular. Recently, there has been a growing interest in the study of Newton's method for solving (1.1) and nonlinear equations with singular solutions [1,3,[5][6][7][13][14][15][16]. These methods retain local quadratic convergence even if the problem has singular solutions.…”

Section: Assumption Amentioning

confidence: 99%

“…For simplicity, in Table 2, we simply use "1" and "2" to denote TRN method and RN method respectively. We define a ratio Ratio Time 1 Time 2 to compare the efficiency of the two methods, where Time 1 and Time 2 denote the cpu time used by TRN method and RN method. The value of Ratio greater than 1 shows that the TRN method performs better than the RN method does.…”

Section: Numerical Experimentsmentioning

confidence: 99%

Truncated regularized Newton method for convex minimizations

2007

Comput Optim Appl

View full text Add to dashboard Cite

show abstract

“…Dan et al [4] have shown that the LM-algorithm is superlinearly convergent if k r ¼ Fðx r Þ k k d ; 0\d62: Furthermore, they showed that LM is quadratically convergent if d = 2. We use the updating formula k r ¼ Fðx r Þ k k 2 [33].…”

Section: Multivariate Minimization Algorithms For Network Trainingmentioning

confidence: 99%

Geno-mathematical identification of the multi-layer perceptron

Östermark

2008

Neural Comput & Applic

View full text Add to dashboard Cite

In this paper, we will focus on the use of the three-layer backpropagation network in vector-valued time series estimation problems. The neural network provides a framework for noncomplex calculations to solve the estimation problem, yet the search for optimal or even feasible neural networks for stochastic processes is both time consuming and uncertain. The backpropagation algorithmwritten in strict ANSI C-has been implemented as a standalone support library for the genetic hybrid algorithm (GHA) running on any sequential or parallel main frame computer. In order to cope with ill-conditioned time series problems, we extended the original backpropagation algorithm to a K nearest neighbors algorithm (K-NARX), where the number K is determined genetically along with a set of key parameters. In the K-NARX algorithm, the terminal solution at instant t can be used as a starting point for the next t, which tends to stabilize the optimization process when dealing with autocorrelated time series vectors. This possibility has proved to be especially useful in difficult time series problems. Following the prevailing research directions, we use a genetic algorithm to determine optimal parameterizations for the network, including the lag structure for the nonlinear vector time series system, the net structure with one or two hidden layers and the corresponding number of nodes, type of activation function (currently the standard logistic sigmoid, a bipolar transformation, the hyperbolic tangent, an exponential function and the sine function), the type of minimization algorithm, the number K of nearest neighbors in the K-NARX procedure, the initial value of the Levenberg-Marquardt damping parameter and the value of the neural learning (stabilization) coefficient a. We have focused on a flexible structure allowing addition of, e.g., new minimization algorithms and activation functions in the future. We demonstrate the power of the genetically trimmed K-NARX algorithm on a representative data set.

show abstract

Convergence Properties of the Inexact Levenberg-Marquardt Method under Local Error Bound Conditions

Cited by 93 publications

References 5 publications

A Derivative-Free Algorithm for Least-Squares Minimization

A Derivative-Free Algorithm for Least-Squares Minimization

Truncated regularized Newton method for convex minimizations

Geno-mathematical identification of the multi-layer perceptron

Contact Info

Product

Resources

About