A primal–dual interior-point framework for using the L1 or L2 norm on the data and regularization terms of inverse problems

Abstract

Maximum a posteriori estimates in inverse problems are often based on quadratic formulations, corresponding to a least-squares fitting of the data and to the use of the L2 norm on the regularization term. While the implementation of this estimation is straightforward and usually based on the Gauss-Newton method, resulting estimates are sensitive to outliers and result in spatial distributions of the estimates that are smooth. As an alternative, the use of the L1 norm on the data term renders the estimation robust to outliers, and the use of the L1 norm on the regularization term allows the reconstruction of sharp spatial profiles. The ability therefore to use the L1 norm either on the data term, on the regularization term, or on both is desirable, though the use of this norm results in non-smooth objective functions which require more sophisticated implementations compared to quadratic algorithms. Methods for L1-norm minimization have been studied in a number of contexts, including in the recently popular total variation regularization. Different approaches have been used and methods based on primal-dual interior-point methods (PD-IPMs) have been shown to be particularly efficient. In this paper we derive a PD-IPM framework for using the L1 norm indifferently on the two terms of an inverse problem. We use electrical impedance tomography as an example inverse problem to demonstrate the implementation of the algorithms we derive, and the effect of choosing the L2 or the L1 norm on the two terms of the inverse problem. Pseudo-codes for the algorithms and a public domain implementation are provided.

Stochastic Inversion of Geomagnetic Observatory Data Including Rigorous Treatment of the Ocean Induction Effect With Implications for Transition Zone Water Content and Thermal Structure

“…Alternatively, deterministic derivative-based methods, such as quasi-Newton or Gauss-Newton, can be applied. However, for nonlinear inverse problems, these methods are highly dependent on the initial model, typically converge to a local minimum, require additional adjustments (e.g., Borsic & Adler, 2012) in the case of non-L 2 norms , and do not provide uncertainty quantification. Therefore, our inverse problem formulation is based on a mixed scheme that combines a stochastic optimization technique known as Covariance Matrix Adaptation Evolution Strategy (CMAES) (Hansen & Ostermeier, 2001), with McMC methods (e.g., Mosegaard & Tarantola, 1995).…”

Magnetic Resonance-Based Electrical Property Tomography (MR-EPT) for Prostate Cancer Grade Imaging

“…TV regularization seems in this case to suffer less from the staircase effect compared to application in other tomographic applications. For example in Electrical Impedance Tomography the staircase effect seems to be more pronounced [25]. We believe this might be due to the fact that in MR-EPT data is measured everywhere in the domain and not only at the boundary as in other techniques.…”

“…In this case, a Primal Dual-Interior Point framework developed in [25], [26] for optimizing (17) was used. Specifically, the algorithm named "PD-IPM-L2-L1 Norm" reported in [25] is used.…”

“…In this case, a Primal Dual-Interior Point framework developed in [25], [26] for optimizing (17) was used. Specifically, the algorithm named "PD-IPM-L2-L1 Norm" reported in [25] is used. The MR-EPT forward model and Jacobian matrix developed in Section IV-A are input into the optimization algorithm, resulting in as defined in (17).…”

Multifrequency electrical impedance tomography with total variation regularization

“…The basic idea underlying PDIPM is the conversion of non-differentiable problems into equivalent differentiable problems by introducing dual variables and the corresponding dual problem (Borsic et al,. 2010;Borsic & Adler, 2012;Fan & Wang, 2010;Mamatjan et al, 2013).…”