Score-Based Parameter Estimation for a Class of Continuous-Time State Space Models

Beskos, Alexandros; Crisan, Dan; Jasra, Ajay; Kantas, Nikolas; Ruzayqat, Hamza

doi:10.1137/20m1362942

Cited by 8 publications

(7 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The following result, which establishes the convergence of our Euler approximation, is proved in [3]. Note that the rate should be O(∆ l ), however, this is not important in the subsequent development of this paper.…”

Section: Discretized Modelmentioning

confidence: 85%

“…Proposition 2.1. Assume (D1-3) in [3]. Then, for any (T, θ) ∈ [0, ∞) × Θ, there exists C < +∞ such that for any l ≥ 0, we have…”

Section: Discretized Modelmentioning

confidence: 99%

“…We use a particular identity for the score function that is provided in [9] and based on the Girsanov change of measure. Alternative identities are discussed in [3] but are not considered in this paper.…”

Section: Introductionmentioning

confidence: 99%

“…In the context of state-space models in discrete and continuous time, there already exists substantial literature on score estimation (see, e.g., [3,7,8,21]). Most of these techniques are based on sequential Monte Carlo (SMC) algorithms (see [11] for an introduction), which are simulation-based methods that use a collection of N ≥ 1 samples generated in parallel and sequentially in time.…”

Section: Introductionmentioning

confidence: 99%

“…We assume throughout this paper that T ∈ N. Note that [3] derives an alternative expression to (4) that does not require σ to be independent of θ; however, its approximation is significantly more complex than we consider.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Unbiased Estimation of the Gradient of the Log-Likelihood for a Class of Continuous-Time State-Space Models

Ballesio¹,

Jasra²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper, we consider static parameter estimation for a class of continuous-time state-space models. Our goal is to obtain an unbiased estimate of the gradient of the log-likelihood (score function), which is an estimate that is unbiased even if the stochastic processes involved in the model must be discretized in time.To achieve this goal, we apply a doubly randomized scheme (see, e.g., [13,14]), that involves a novel coupled conditional particle filter (CCPF) on the second level of randomization [15]. Our novel estimate helps facilitate the application of gradient-based estimation algorithms, such as stochastic-gradient Langevin descent. We illustrate our methodology in the context of stochastic gradient descent (SGD) in several numerical examples and compare with the Rhee & Glynn estimator [22,23].

show abstract

Section: Discretized Modelmentioning

confidence: 85%

“…Proposition 2.1. Assume (D1-3) in [3]. Then, for any (T, θ) ∈ [0, ∞) × Θ, there exists C < +∞ such that for any l ≥ 0, we have…”

Section: Discretized Modelmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Unbiased Estimation of the Gradient of the Log-Likelihood for a Class of Continuous-Time State-Space Models

Ballesio¹,

Jasra²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Multi-index Sequential Monte Carlo Ratio Estimators for Bayesian Inverse problems

et al. 2023

View full text Add to dashboard Cite

We consider the problem of estimating expectations with respect to a target distribution with an unknown normalising constant, and where even the un-normalised target needs to be approximated at finite resolution. This setting is ubiquitous across science and engineering applications, for example in the context of Bayesian inference where a physics-based model governed by an intractable partial differential equation (PDE) appears in the likelihood. A multi-index sequential Monte Carlo (MISMC) method is used to construct ratio estimators which provably enjoy the complexity improvements of multi-index Monte Carlo (MIMC) as well as the efficiency of sequential Monte Carlo (SMC) for inference. In particular, the proposed method provably achieves the canonical complexity of $$\hbox {MSE}^{-1}$$ MSE - 1 , while single-level methods require $$\hbox {MSE}^{-\xi }$$ MSE - ξ for $$\xi >1$$ ξ > 1 . This is illustrated on examples of Bayesian inverse problems with an elliptic PDE forward model in 1 and 2 spatial dimensions, where $$\xi =5/4$$ ξ = 5 / 4 and $$\xi =3/2$$ ξ = 3 / 2 , respectively. It is also illustrated on more challenging log-Gaussian process models, where single-level complexity is approximately $$\xi =9/4$$ ξ = 9 / 4 and multilevel Monte Carlo (or MIMC with an inappropriate index set) gives $$\xi = 5/4 + \omega $$ ξ = 5 / 4 + ω , for any $$\omega > 0$$ ω > 0 , whereas our method is again canonical. We also provide novel theoretical verification of the product-form convergence results which MIMC requires for Gaussian processes built in spaces of mixed regularity defined in the spectral domain, which facilitates acceleration with fast Fourier transform methods via a cumulant embedding strategy, and may be of independent interest in the context of spatial statistics and machine learning.

show abstract

Multilevel estimation of normalization constants using ensemble Kalman–Bucy filters

2022

View full text Add to dashboard Cite

Score-Based Parameter Estimation for a Class of Continuous-Time State Space Models

Cited by 8 publications

References 24 publications

Unbiased Estimation of the Gradient of the Log-Likelihood for a Class of Continuous-Time State-Space Models

Unbiased Estimation of the Gradient of the Log-Likelihood for a Class of Continuous-Time State-Space Models

Multi-index Sequential Monte Carlo Ratio Estimators for Bayesian Inverse problems

Multilevel estimation of normalization constants using ensemble Kalman–Bucy filters

Contact Info

Product

Resources

About