The Impact of the Discrepancy Principle on the Tikhonov-Regularized Solutions with Oversmoothing Penalties

Hofmann, Bernd; Hofmann, Christopher

doi:10.3390/math8030331

Cited by 7 publications

(10 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The basis for an analytical treatment of the balancing principle is formed by general error estimates for Tikhonov regularization in Hilbert scales with oversmoothing penalty. With the inequality chain (3) and for x † ∈ int(D(F )) such estimates have been developed recently in [9], [13], and [7] by using auxiliary elements as minimizers of (4). Introducing the injective linear operator G := B −(2a+2) , the corresponding minimizers xα can be expressed explicitly as…”

Section: General Error Estimate For Tikhonov Regularization In Hilber...mentioning

confidence: 99%

“…Proof. A substantial ingredient of the proof is the fact that due to [19] sub-linear index functions are qualifications for the classical Tikhonov regularization approach with norm square penalty, and the verification of formula ( 25) in [7], which in turn is based on the bounds ( 21)-( 23) ibid. As can be seen from there it is enough to show that under the above assumptions, and for 0 ≤ θ ≤ 2a+1 2a+2 , we have that ( 7)…”

Section: Error Decompositionmentioning

confidence: 99%

“…It is highlighted there that such error decomposition also allows for low order convergence rates under low order smoothness assumptions on x † . But it was observed in [7] that the error decomposition from [13] extends to power type smoothness x † ∈ X p , 0 < p < 1, and hence yields optimal rates of convergence under the a priori parameter choice. 1.4.…”

Section: Introductionmentioning

confidence: 99%

“…On the one hand side, it extends the error decomposition from [13,7] to more general smoothness assumptions. On the other hand, we present new results to the balancing principle for choosing the regularization parameter in Tikhonov regularization for nonlinear problems with oversmoothing penalties in a Hilbert scale setting.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Nonlinear Tikhonov regularization in Hilbert scales with oversmoothing penalty: inspecting balancing principles

Hofmann¹,

Hofmann²,

Mathé³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

The analysis of Tikhonov regularization for nonlinear ill-posed equations with smoothness promoting penalties is an important topic in inverse problem theory. With focus on Hilbert scale models, the case of oversmoothing penalties, i.e., when the penalty takes an infinite value at the true solution gained increasing interest. The considered nonlinearity structure is as in the study B. Hofmann and P. Mathé. Tikhonov regularization with oversmoothing penalty for non-linear ill-posed problems in Hilbert scales. Inverse Problems, 2018. Such analysis can address two fundamental questions. When is it possible to achieve order optimal reconstruction? How to select the regularization parameter? The present study complements previous ones by two main facets. First, an error decomposition into a smoothness dependent and a (smoothness independent) noise propagation term is derived, covering a large range of smoothness conditions. Secondly, parameter selection by balancing principles is presented. A detailed discussion, covering some history and variations of the parameter choice by balancing shows under which conditions such balancing principles yield order optimal reconstruction. A numerical case study, based on some exponential growth model, provides additional insights. 1 and consequently x δ α ∈ D(B) = X 1 for all data y δ ∈ Y and α > 0. In order to ensure existence and stability of the regularized solutions x δ α for all α > 0 (cf. [8, § 3], [25, Section 3.2] and [26, Section 4.1.1]), we additionally suppose that the forward operator F is weakly sequentially continuous.

show abstract

Section: General Error Estimate For Tikhonov Regularization In Hilber...mentioning

confidence: 99%

Section: Error Decompositionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Nonlinear Tikhonov regularization in Hilbert scales with oversmoothing penalty: inspecting balancing principles

Hofmann¹,

Hofmann²,

Mathé³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the case when L is based on a norm, such as L(x, y) = 1 2 x − y 2 2 , conditions on Φ to guarantee the existence of a minimum to (3.1) are given in [34,Thm. 4.1] and [22,Prop. 2].…”

mentioning

confidence: 99%

Learning via nonlinear conjugate gradients and depth-varying neural ODEs

Baravdish¹,

Eilertsen²,

Jaroudi³

et al. 2022

Preprint

View full text Add to dashboard Cite

The inverse problem of supervised reconstruction of depth-variable (time-dependent) parameters in a neural ordinary differential equation (NODE) is considered, that means finding the weights of a residual network with time continuous layers. The NODE is treated as an isolated entity describing the full network as opposed to earlier research, which embedded it between pre-and post-appended layers trained by conventional methods. The proposed parameter reconstruction is done for a general first order differential equation by minimizing a cost functional covering a variety of loss functions and penalty terms. A nonlinear conjugate gradient method (NCG) is derived for the minimization. Mathematical properties are stated for the differential equation and the cost functional. The adjoint problem needed is derived together with a sensitivity problem. The sensitivity problem can estimate changes in the network output under perturbation of the trained parameters. To preserve smoothness during the iterations the Sobolev gradient is calculated and incorporated. As a proof-of-concept, numerical results are included for a NODE and two synthetic datasets, and compared with standard gradient approaches (not based on NODEs). The results show that the proposed method works well for deep learning with infinite numbers of layers, and has built-in stability and smoothness.

show abstract

A Hybrid Sobolev Gradient Method for Learning NODEs

Baravdish,

Eilertsen,

Jaroudi

et al. 2024

Oper. Res. Forum

View full text Add to dashboard Cite

The inverse problem of supervised reconstruction of depth-variable (time-dependent) parameters in ordinary differential equations is considered, with the typical application of finding weights of a neural ordinary differential equation (NODE) for a residual network with time continuous layers. The differential equation is treated as an abstract and isolated entity, termed a standalone NODE (sNODE), to facilitate for a wide range of applications. The proposed parameter reconstruction is performed by minimizing a cost functional covering a variety of loss functions and penalty terms. Regularization via penalty terms is incorporated to enhance ethical and trustworthy AI formulations. A nonlinear conjugate gradient mini-batch optimization scheme (NCG) is derived for the training having the benefit of including a sensitivity problem. The model (differential equation)-based approach is thus combined with a data-driven learning procedure. Mathematical properties are stated for the differential equation and the cost functional. The adjoint problem needed is derived together with the sensitivity problem. The sensitivity problem itself can estimate changes in the output under perturbation of the trained parameters. To preserve smoothness during the iterations, the Sobolev gradient is calculated and incorporated. Numerical results are included to validate the procedure for a NODE and synthetic datasets and compared with standard gradient approaches. For stability, using the sensitivity problem, a strategy for adversarial attacks is constructed, and it is shown that the given method with Sobolev gradients is more robust than standard approaches for parameter identification.

show abstract

The Impact of the Discrepancy Principle on the Tikhonov-Regularized Solutions with Oversmoothing Penalties

Cited by 7 publications

References 29 publications

Nonlinear Tikhonov regularization in Hilbert scales with oversmoothing penalty: inspecting balancing principles

Nonlinear Tikhonov regularization in Hilbert scales with oversmoothing penalty: inspecting balancing principles

Learning via nonlinear conjugate gradients and depth-varying neural ODEs

A Hybrid Sobolev Gradient Method for Learning NODEs

Contact Info

Product

Resources

About