A new look at entropy for solving linear inverse problems

Besnerais, Guy Le; Bercher, Jean‐François; Demoment, G.

doi:10.1109/18.771159

Cited by 43 publications

(44 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, large deviation results (Ellis, 1985;Le Besnerais et al, 1999) of pdfs strongly support this particular form, on theoretical grounds. Indeed, the exponential of minus the relative entropy is known to be the a measure of the deviation around the prior law, in the large deviation limit.…”

Section: Level-one Primal Cost Functionmentioning

confidence: 99%

Inverse modelling of atmospheric tracers: non-Gaussian methods and second-order sensitivity analysis

Bocquet

2008

Nonlin. Processes Geophys.

View full text Add to dashboard Cite

Abstract. For a start, recent techniques devoted to the reconstruction of sources of an atmospheric tracer at continental scale are introduced. A first method is based on the principle of maximum entropy on the mean and is briefly reviewed here. A second approach, which has not been applied in this field yet, is based on an exact Bayesian approach, through a maximum a posteriori estimator. The methods share common grounds, and both perform equally well in practice. When specific prior hypotheses on the sources are taken into account such as positivity, or boundedness, both methods lead to purposefully devised cost-functions. These cost-functions are not necessarily quadratic because the underlying assumptions are not Gaussian. As a consequence, several mathematical tools developed in data assimilation on the basis of quadratic cost-functions in order to establish a posteriori analysis, need to be extended to this nonGaussian framework. Concomitantly, the second-order sensitivity analysis needs to be adapted, as well as the computations of the averaging kernels of the source and the errors obtained in the reconstruction. All of these developments are applied to a real case of tracer dispersion: the European Tracer Experiment [ETEX]. Comparisons are made between a least squares cost function (similar to the so-called 4D-Var) approach and a cost-function which is not based on Gaussian hypotheses. Besides, the information content of the observations which is used in the reconstruction is computed and studied on the application case. A connection with the degrees of freedom for signal is also established. As a byproduct of these methodological developments, conclusions are drawn on the information content of the ETEX dataset as seen from the inverse modelling point of view.

show abstract

Section: Level-one Primal Cost Functionmentioning

confidence: 99%

Inverse modelling of atmospheric tracers: non-Gaussian methods and second-order sensitivity analysis

Bocquet

2008

Nonlin. Processes Geophys.

View full text Add to dashboard Cite

show abstract

“…Indeed, it can be shown that when we optimize a convex criterion subject to the data constraints we are optimizing the entropy of some quantity related to the unknowns and vise versa. However, as we have mentioned, basically, in this approach the data and the model are assumed to be exact even if some extensions to the approach gives the possibility to account for the errors [9]. In the next section, we see how the Bayesian approach can naturally account for both uncertainties on the data and on the unknown parameters x.…”

Section: Maximum Entropy In the Meanmentioning

confidence: 99%

Bayesian inference for inverse problems

Mohammad‐Djafari

2002

AIP Conference Proceedings

View full text Add to dashboard Cite

Abstract. Traditionally, the MaxEnt workshops start by a tutorial day. This paper summarizes my talk during 2001'th workshop at John Hopkins University. The main idea in this talk is to show how the Bayesian inference can naturally give us all the necessary tools we need to solve real inverse problems: starting by simple inversion where we assume to know exactly the forward model and all the input model parameters up to more realistic advanced problems of myopic or blind inversion where we may be uncertain about the forward model and we may have noisy data.Starting by an introduction to inverse problems through a few examples and explaining their ill posedness nature, I briefly presented the main classical deterministic methods such as data matching and classical regularization methods to show their limitations. I then presented the main classical probabilistic methods based on likelihood, information theory and maximum entropy and the Bayesian inference framework for such problems. I show that the Bayesian framework, not only generalizes all these methods, but also gives us natural tools, for example, for inferring the uncertainty of the computed solutions, for the estimation of the hyperparameters or for handling myopic or blind inversion problems. Finally, through a deconvolution problem example, I presented a few state of the art methods based on Bayesian inference particularly designed for some of the mass spectrometry data processing problems.

show abstract

“…If, on the other hand, the problem is ill-posed (e.g., X is not invertible), then the solution is not unique, and a combination of the above two methods (16 and 18) can be used. This yields the regularization method consisting of finding E such that: (20) is achieved (see for example, Donoho et al [25] for a nice discussion of regularization within the ME formulation.) Traditionally, the positive penalization parameter D is specified to favor small sized reconstructions, meaning that out of all possible reconstructions with a given discrepancy, those with the smallest norms are chosen.…”

Section: The General Casementioning

confidence: 99%

“…See also Gzyl and Velásquez [2], which builds upon Golan and Gzyl [18] where the synthesis was first proposed. If, in addition, the data are ill-conditioned, one often has to resort to the class of regularization methods (e.g., Hoerl and Kennard [19] O'Sullivan [20], Breiman [21], Tibshirani [22], Titterington [23], Donoho et al [24]; Besnerais et al [25]. A reference for regularization in statistics is Bickel and Li [26].…”

Section: Introductionmentioning

confidence: 99%

An Entropic Estimator for Linear Inverse Problems

Golan

Gzyl

2012

Entropy

View full text Add to dashboard Cite

In this paper we examine an Information-Theoretic method for solving noisy linear inverse estimation problems which encompasses under a single framework a whole class of estimation methods. Under this framework, the prior information about the unknown parameters (when such information exists), and constraints on the parameters can be incorporated in the statement of the problem. The method builds on the basics of the maximum entropy principle and consists of transforming the original problem into an estimation of a probability density on an appropriate space naturally associated with the statement of the problem. This estimation method is generic in the sense that it provides a framework for analyzing non-normal models, it is easy to implement and is suitable for all types of inverse problems such as small and or ill-conditioned, noisy data. First order approximation, large sample properties and convergence in distribution are developed as well. Analytical examples, statistics for model comparisons and evaluations, that are inherent to this method, are discussed and complemented with explicit examples.

show abstract

A new look at entropy for solving linear inverse problems

Cited by 43 publications

References 39 publications

Inverse modelling of atmospheric tracers: non-Gaussian methods and second-order sensitivity analysis

Inverse modelling of atmospheric tracers: non-Gaussian methods and second-order sensitivity analysis

Bayesian inference for inverse problems

An Entropic Estimator for Linear Inverse Problems

Contact Info

Product

Resources

About