The linear conditional expectation in Hilbert space

Klebanov, Ilja; Sprungk, Björn; Sullivan, Tim

doi:10.3150/20-bej1308

Cited by 5 publications

(7 citation statements)

References 31 publications

(52 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All other solutions of (RP) perform operations on ker(C XX ) which are irrelevant for the regression. (c) As also noted by Klebanov et al (2021), it is easy to see that the equivalent conditions given in Proposition 3.11 are always satisfied whenever X is finite-dimensional, as in this case C † XX is always bounded.…”

Section: Existence Of the Minimiser θ ⋆ ∈ L(x Y)mentioning

confidence: 77%

“…This confirms Proposition 3.5 in the setting that the linear regression problem is well-specified in the context of statistical learning theory. Note that the linear model above is equivalent to assuming the validity of the linear conditional expectation (Klebanov et al, 2021) representation…”

Section: Existence Of the Minimiser θ ⋆ ∈ L(x Y)mentioning

confidence: 99%

“…where C YX ∈ L(X , Y) and C XX ∈ L(X , X ) are the covariance operators (Baker, 1973) associated with X and Y , which is again related to an well-known set of range inclusion and operator majorisation conditions due to Douglas (1966) and the Moore-Penrose pseudoinverse (Engl et al, 1996). The representation of linear conditional expectations in terms of pseudoinverses of covariance operators has also been investigated recently by Klebanov et al (2021).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Learning linear operators: Infinite-dimensional regression as a well-behaved non-compact inverse problem

Mollenhauer¹,

Mücke²,

Sullivan³

2022

Preprint

View full text Add to dashboard Cite

We consider the problem of learning a linear operator θ between two Hilbert spaces from empirical observations, which we interpret as least squares regression in infinite dimensions. We show that this goal can be reformulated as an inverse problem for θ with the undesirable feature that its forward operator is generally non-compact (even if θ is assumed to be compact or of p-Schatten class). However, we prove that, in terms of spectral properties and regularisation theory, this inverse problem is equivalent to the known compact inverse problem associated with scalar response regression.Our framework allows for the elegant derivation of dimension-free rates for generic learning algorithms under Hölder-type source conditions. The proofs rely on the combination of techniques from kernel regression with recent results on concentration of measure for subexponential Hilbertian random variables. The obtained rates hold for a variety of practicallyrelevant scenarios in functional regression as well as nonlinear regression with operatorvalued kernels and match those of classical kernel regression with scalar response.

show abstract

Section: Existence Of the Minimiser θ ⋆ ∈ L(x Y)mentioning

confidence: 77%

Section: Existence Of the Minimiser θ ⋆ ∈ L(x Y)mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning linear operators: Infinite-dimensional regression as a well-behaved non-compact inverse problem

Mollenhauer¹,

Mücke²,

Sullivan³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Deriving the Wiener filter, in this case, amounts to deriving the BLUE for X given Y . In Hilbert space, this corresponds to linear conditional expectation (LCE) (see Section SIII of the supplementary or [41]). These results enable us to derive an explicit formula for the Wiener filter.…”

Section: A Wiener Filter For Denoisingmentioning

confidence: 99%

Wide-Sense Stationarity in Generalized Graph Signal Processing

Jian

Tay

2022

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

We consider statistical graph signal processing (GSP) in a generalized framework where each vertex of a graph is associated with an element from a Hilbert space. This general model encompasses various signals such as the traditional scalar-valued graph signal, multichannel graph signal, and discrete-and continuous-time graph signals, allowing us to build a unified theory of graph random processes. We introduce the notion of joint wide-sense stationarity in this generalized GSP framework, which allows us to characterize a graph random process as a combination of uncorrelated oscillation modes across both the vertex and Hilbert space domains. We elucidate the relationship between the notions of wide-sense stationarity in different domains, and derive the Wiener filters for denoising and signal completion under this framework. Numerical experiments on both real and synthetic datasets demonstrate the utility of our generalized approach in achieving better estimation performance compared to traditional GSP or the time-vertex framework.

show abstract

“…Gaussian processes (GP) provide a powerful Bayesian approach to regression (Rasmussen and Williams, 2006). While traditional regression considers pointwise evaluations of an unknown function, GPs can also include data in the form of linear operators (Solak et al, 2003;Särkkä, 2011a;Jidling et al, 2017;Mandelbaum, 1984;Tarieladze and Vakhania, 2007;Hairer et al, 2005;Owhadi and Scovel, 2015;Klebanov et al, 2020). This allows GPs to provide a Bayesian framework to address inverse problems (Tarantola and Valette, 1982;Stuart, 2010;Dashti and Stuart, 2016).…”

mentioning

confidence: 99%

Uncertainty Quantification and Experimental Design for Large-Scale Linear Inverse Problems under Gaussian Process Priors

Cédric¹,

Ginsbourger²,

Linde³

2021

Preprint

View full text Add to dashboard Cite

We consider the use of Gaussian process (GP) priors for solving inverse problems in a Bayesian framework. As is well known, the computational complexity of GPs scales cubically in the number of datapoints. We here show that in the context of inverse problems involving integral operators, one faces additional difficulties that hinder inversion on large grids. Furthermore, in that context, covariance matrices can become too large to be stored. By leveraging results about sequential disintegrations of Gaussian measures, we are able to introduce an implicit representation of posterior covariance matrices that reduces the memory footprint by only storing low rank intermediate matrices, while allowing individual elements to be accessed on-the-fly without needing to build full posterior covariance matrices. Moreover, it allows for fast sequential inclusion of new observations. These features are crucial when considering sequential experimental design tasks. We demonstrate our approach by computing sequential data collection plans for excursion set recovery for a gravimetric inverse problem, where the goal is to provide fine resolution estimates of high density regions inside the Stromboli volcano, Italy. Sequential data collection plans are computed by extending the weighted integrated variance reduction (wIVR) criterion to inverse problems. Our results show that this criterion is able to significantly reduce the uncertainty on the excursion volume, reaching close to minimal levels of residual uncertainty. Overall, our techniques allow the advantages of probabilistic models to be brought to bear on large-scale inverse problems arising in the natural sciences. Particularly, applying the latest developments in Bayesian sequential experimental design on realistic large-scale problems opens new venues of research at a crossroads between mathematical modelling of natural phenomena, statistical data science and active learning.

show abstract

The linear conditional expectation in Hilbert space

Cited by 5 publications

References 31 publications

Learning linear operators: Infinite-dimensional regression as a well-behaved non-compact inverse problem

Learning linear operators: Infinite-dimensional regression as a well-behaved non-compact inverse problem

Wide-Sense Stationarity in Generalized Graph Signal Processing

Uncertainty Quantification and Experimental Design for Large-Scale Linear Inverse Problems under Gaussian Process Priors

Contact Info

Product

Resources

About