Regularization Methods for Uniformly Rank-Deficient Nonlinear Least-Squares Problems

Eriksson, Jerry; Wedin, Per-Åke; Gulliksson, Mårten; Söderkvist, Inge

doi:10.1007/s10957-005-6389-0

Cited by 22 publications

(16 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The idea of constructing an iterative method for the computation of the minimal norm solution of a nonlinear least-squares problem was first studied by Eriksson et al In [15,16,17], the case where the Jacobian is rank-deficient or ill-conditioned was analyzed, and solution techniques based on the Gauss-Newton method and on Tikhonov regularization in standard form were proposed.…”

Section: R M (X)]mentioning

confidence: 99%

“…We will explore in this paper what the consequence is of imposing a regularity constraint directly on the solution x of problem (1.1). Approaches of this kind were studied by Eriksson and Wedin [15,16,17]: they proposed a minimal-norm Gauss-Newton method and a Tikhonov regularization method in standard form. We extend, in Theorem 4.2, the minimal-norm Gauss-Newton method by introducing a regularization matrix L. Moreover, in Section 5 we investigate Tikhonov regularization in general form and the use of truncated SVD/GSVD in the minimal-norm Gauss-Newton method.…”

mentioning

confidence: 99%

See 1 more Smart Citation

The minimal-norm Gauss-Newton method and some of its regularized variants

Pes¹,

Rodriguez²

2020

etna

View full text Add to dashboard Cite

Nonlinear least-squares problems appear in many real-world applications. When a nonlinear model is used to reproduce the behavior of a physical system, the unknown parameters of the model can be estimated by fitting experimental observations by a least-squares approach. It is common to solve such problems by Newton's method or one of its variants such as the Gauss-Newton algorithm. In this paper, we study the computation of the minimal-norm solution to a nonlinear least-squares problem, as well as the case where the solution minimizes a suitable semi-norm. Since many important applications lead to severely ill-conditioned least-squares problems, we also consider some regularization techniques for their solution. Numerical experiments, both artificial and derived from an application in applied geophysics, illustrate the performance of the different approaches.

show abstract

Section: R M (X)]mentioning

confidence: 99%

mentioning

confidence: 99%

The minimal-norm Gauss-Newton method and some of its regularized variants

Pes¹,

Rodriguez²

2020

etna

View full text Add to dashboard Cite

show abstract

“…Output: Solution to f (x) = 0 or if k = k max the vector x kmax A restart, see step 14., is performed if either the step length is too small indicating not enough descent, or if the gradient is small while the norm of f is not small, see step 12. The first case appears when the Gauss-Newton method does not converge locally, i.e., the solution has a large residual f and/or a small curvature, see [17] for details. The second case for a restart may occur when the algorithm is converging to a local minima where the norm of f is not close to zero.…”

Section: Greedy Gauss-newton Algorithmmentioning

confidence: 99%

Greedy Gauss-Newton algorithms for finding sparse solutions to nonlinear underdetermined systems of equations

Gulliksson

Oleynik

2017

Optimization

View full text Add to dashboard Cite

We consider the problem of finding sparse solutions to a system of underdetermined nonlinear system of equations. The methods are based on a Gauss-Newton approach with line search where the search direction is found by solving a linearized problem using only a subset of the columns in the Jacobian. The choice of columns in the Jacobian is made through a greedy approach looking at either maximum descent or an approach corresponding to orthogonal matching for linear problems. The methods are shown to be convergent and efficient and outperform the 1 approach on the test problems presented.2000 Mathematics Subject Classification. 68Q25, 68R10, 68U05.

show abstract

“…Many practical problems can be converted into solving this problem, such as ill-posed problems, inverse problems, some constrained optimization problems, and model parameter estimation [27,31,28,29,14,15].…”

Section: Numerical Experimentsmentioning

confidence: 99%

Global Convergence of a New Hybrid Gauss–Newton Structured BFGS Method for Nonlinear Least Squares Problems

Zhou¹,

Chen²

2010

SIAM J. Optim.

View full text Add to dashboard Cite

Abstract. In this paper, we propose a hybrid Gauss-Newton structured BFGS method with a new update formula and a new switch criterion for the iterative matrix to solve nonlinear least squares problems. We approximate the second term in the Hessian by a positive definite BFGS matrix. Under suitable conditions, global convergence of the proposed method with a backtracking line search is established. Moreover, the proposed method automatically reduces to the GaussNewton method for zero residual problems and the structured BFGS method for nonzero residual problems in a neighborhood of an accumulation point. A locally quadratic convergence rate for zero residual problems and a locally superlinear convergence rate for nonzero residual problems are obtained for the proposed method. Some numerical results are given to compare the proposed method with some existing methods.

show abstract

Regularization Methods for Uniformly Rank-Deficient Nonlinear Least-Squares Problems

Cited by 22 publications

References 8 publications

The minimal-norm Gauss-Newton method and some of its regularized variants

The minimal-norm Gauss-Newton method and some of its regularized variants

Greedy Gauss-Newton algorithms for finding sparse solutions to nonlinear underdetermined systems of equations

Global Convergence of a New Hybrid Gauss–Newton Structured BFGS Method for Nonlinear Least Squares Problems

Contact Info

Product

Resources

About