A Newton root-finding algorithm for estimating the regularization parameter for solving ill-conditioned least squares problems

Mead, Jodi; Renaut, Rosemary A.

doi:10.1088/0266-5611/25/2/025002

Cited by 40 publications

(73 citation statements)

References 22 publications

(67 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is an extension of the scalar χ 2 method [21,22,23,31] which can be viewed as a regularization method. The new method amounts to solving multiple χ 2 tests to give an equal number of equations as the number of unknowns in the diagonal weighting matrix for data or parameter misfits.…”

Section: Discussionmentioning

confidence: 99%

“…This statistical interpretation of the weights in (3) and multipliers in (8) gives us a method for calculating them, and we term it the χ 2 method for parameter estimation and uncertainty quantification [21,22,23,31].…”

Section: Regularization and Constrained Optimizationmentioning

confidence: 99%

“…In Section 3 we describe a method to find weights which give piecewise smooth least squares estimates. This method is an extension of the χ 2 method [21,22], and we illustrate its effectiveness in finding discontinuous parameters by solving a benchmark ill-conditioned problem in [15]. In Section 4 we use the method to find matrix weights that quantify uncertainty when estimating water content in soils.…”

Section: Introductionmentioning

confidence: 99%

“…Algorithms for the χ 2 method when C f = σ 2 f I and C = σ 2 I are given in [22,31], and we refer to this case as the scalar χ 2 method. This is Tikohonv regularization [39] and it was shown that the scalar χ 2 method is an attractive alternative to the L-curve [13] and Generalized Cross Validation (GCV) [46].…”

mentioning

confidence: 99%

“…Issues with these latter methods include the fact that the L-curve method is computationally expensive because the problem is solved with many different values of α to determine the optimal value, while it is not always possible to find a satisfactory choice of α that minimizes the GCV functional [40]. The scalar χ 2 method does not have these issues because it finds an optimal α in about 5 Newton iterations, and the functional that it minimizes is monotonically decreasing [22]. A description of GCV and other statistical techniques for choosing α can be found in [41,42].…”

mentioning

confidence: 99%

See 4 more Smart Citations

Discontinuous parameter estimates with least squares estimators

Mead

2013

Applied Mathematics and Computation

View full text Add to dashboard Cite

We discuss weighted least squares estimates of ill-conditioned linear inverse problems where weights are chosen to be inverse error covariance matrices. Least squares estimators are the maximum likelihood estimate for normally distributed data and parameters, but here we do not assume particular probability distributions. Weights for the estimator are found by ensuring its minimum follows a χ 2 distribution. Previous work with this approach has shown that the it is competitive with regularization methods such as the L-curve and Generalized Cross Validation (GCV) [23]. In this work we extend the method to find diagonal weighting matrices, rather than a scalar regularization parameter. Diagonal weighting matrices are advantageous because they give piecewise smooth least squares estimates and hence are a mechanism through which least squares can be used to estimate discontinuous parameters. This is explained by viewing least squares estimation as a constrained optimization problem. Results with diagonal weighting matrices are given for a benchmark discontinuous inverse problem from [15]. In addition, the method is used to estimate soil moisture from data collected in the Dry Creek Watershed near Boise, Idaho. Parameter estimates are found that combine two different types of measurements, and weighting matrices are found that incorporate uncertainty due to spatial variation so that the parameters can be used over larger scales than those that were measured.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Regularization and Constrained Optimizationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Discontinuous parameter estimates with least squares estimators

Mead

2013

Applied Mathematics and Computation

View full text Add to dashboard Cite

show abstract

Computational methods for image reconstruction

Chung

Ruthotto

2016

NMR in Biomedicine

View full text Add to dashboard Cite

Reconstructing images from indirect measurements is a central problem in many applications, including the subject of this special issue, quantitative susceptibility mapping (QSM). The process of image reconstruction typically requires solving an inverse problem that is ill-posed and large-scale and thus challenging to solve. Although the research field of inverse problems is thriving and very active with diverse applications, in this part of the special issue we will focus on recent advances in inverse problems that are specific to deconvolution problems, the class of problems to which QSM belongs. We will describe analytic tools that can be used to investigate underlying ill-posedness and apply them to the QSM reconstruction problem and the related extensively studied image deblurring problem. We will discuss state-of-the-art computational tools and methods for image reconstruction, including regularization approaches and regularization parameter selection methods. We finish by outlining some of the current trends and future challenges. Copyright © 2016 John Wiley & Sons, Ltd.

show abstract

Efficient estimation of regularization parameters via downsampling and the singular value expansion

et al. 2016

View full text Add to dashboard Cite

Abstract. The solution, x, of the linear system of equations Ax ≈ b arising from the discretization of an ill-posed integral equation with a square integrable kernel H(s, t) is considered. The Tikhonov regularized solution x(λ) is found as the minimizer of}. x(λ) depends on regularization parameter λ that trades off the data fidelity, and on the smoothing norm determined by L. Here we consider the case where L is diagonal and invertible, and employ the Galerkin method to provide the relationship between the singular value expansion and the singular value decomposition for square integrable kernels. The resulting approximation of the integral equation permits examination of the properties of the regularized solution x(λ) independent of the sample size of the data. We prove that estimation of the regularization parameter can be obtained by consistently down sampling the data and the system matrix, leading to solutions of coarse to fine grained resolution. Hence, the estimate of λ for a large problem may be found by downsampling to a smaller problem, or to a set of smaller problems, effectively moving the costly estimate of the regularization parameter to the coarse representation of the problem. Moreover, the full singular value decomposition for the fine scale system is replaced by a number of dominant terms which is determined from the coarse resolution system, again reducing the computational cost. Numerical results illustrate the theory and demonstrate the practicality of the approach for regularization parameter estimation using generalized cross validation, unbiased predictive risk estimation and the discrepancy principle applied for both the system of equations, and the augmented system of equations.

show abstract

A Newton root-finding algorithm for estimating the regularization parameter for solving ill-conditioned least squares problems

Cited by 40 publications

References 22 publications

Discontinuous parameter estimates with least squares estimators

Discontinuous parameter estimates with least squares estimators

Computational methods for image reconstruction

Efficient estimation of regularization parameters via downsampling and the singular value expansion

Contact Info

Product

Resources

About