Sparse Bayesian Estimation of Parameters in Linear-Gaussian State-Space Models

Abstract

State-space models (SSMs) are a powerful statistical tool for modelling time-varying systems via a latent state. In these models, the latent state is never directly observed. Instead, a sequence of data points related to the state are obtained. The linear-Gaussian state-space model is widely used, since it allows for exact inference when all model parameters are known, however this is rarely the case. The estimation of these parameters is a very challenging but essential task to perform inference and prediction. In the linear-Gaussian model, the state dynamics are described via a state transition matrix. This model parameter is known to behard to estimate, since it encodes the relationships between the state elements, which are never observed. In many applications, this transition matrix is sparse since not all state components directly affect all other state components. However, most parameter estimation methods do not exploit this feature. In this work we propose SpaRJ, a fully probabilistic Bayesian approach that obtains sparse samples from the posterior distribution of the transition matrix. Our method explores sparsity by traversing a set of models that exhibit differing sparsity patterns in the transition matrix. Moreover, we also design new effective rules to explore transition matrices within the same level of sparsity. This novel methodology has strong theoretical guarantees, and unveils the latent structure of the data generating process, thereby enhancing interpretability. The performance of SpaRJ is showcased in example with dimension 144 in the parameter space, and in a numerical example with real data.

Graphical Inference in Non-Markovian Linear-Gaussian State-Space Models

“…LaGrangEM can be seen as an SSM with a Granger causality model in its latent process, which enhances interpretability due to the promoted structure and sparsity. Another advantage of LaGrangEM is that it keeps its probabilistic nature in both state and observation models, and could be easily extended to a fully Bayesian approach (e.g., as in SpaRJ [18]). Unlike in Granger-based fitting methods, LaGrangEM allows to promote structure in the latent process by incorporating prior knowledge.…”

“…Existing works based on graphical models time-series include [6][7][8][9], with applications in biology [10,11], networks [12], and neuroscience [13]. Recent works have dealt with parameter estimation in Markovian SSMs through a graphical perspective, either point-wise [14][15][16] or fully probabilistic [17,18].…”

Graphit: Iterative Reweighted ℓ1Algorithm for Sparse Graph Inference in State-Space Models

“…It also allows for the incorporation physical knowledge about the system and to retrieve relevant structure that can be inferred when processing the data. There exists abundant literature in graphical models for multi-variate time series, e.g., [1][2][3][4], and more recently also within SSMs [5,6], including also fully probabilistic approaches [7,8]. There are multiple applications of such graphical representa-V.E.…”