Robust Autoregression: Student-t Innovations Using Variational Bayes

Abstract

Abstract-Autoregression (AR) is a tool commonly used to understand and predict time series data. Traditionally the excitation noise is modelled as a Gaussian. However, real-world data may not be Gaussian in nature, and it is known that Gaussian models are adversely affected by the presence of outliers. We introduce a Bayesian AR model in which the excitation noise is assumed to be Student-t distributed. Variational Bayesian approximations to the posterior distributions of the model parameters are used to overcome the intractable integrations inherent in the Bayesian model. Independent Automatic Relevance Determination (ARD) priors over each of the AR coefficients are used to estimate the model order.Using synthetic data we show that the Student-t model performs well against both Gaussian and leptokurtic data, in terms of parameter estimation (including the model order), and is much more robust to outliers than either Gaussian or finite mixtures of Gaussians models.We apply the model to strongly leptokurtic EEG signals and show that the Student-t model makes more accurate one-stepahead predictions than the Gaussian model and provides more consistent estimates of the AR coefficients over simultaneously recorded EEG channels.

Theoretical motion functions for video analysis, with a passive navigation example

“…One way to suppress them would be to assign them automatic • (for comparison with the map) and start at origin of the plot, but no scaling has been performed. relevance determination (ARD) priors [28,29] (see, for example, [30]), which tends to try and constrain their values to be close to zero (or some other value). In model 4 (29), for example, using an ARD prior might constrain the coefficient for the r 2 n term to be close to zero, and hence this model might achieve the same results as model 2 (28) which does not contain that term.…”

“…The probability p(Y n | F(Y n−1 , x n ), κ n ) was calculated for each frame of the video and for each of the four potential models shown in (27)- (30). Histograms of the resulting values are plotted in figure 7, where the distributions show that model 3 gives the worst estimates, while model 2 gives the best.…”

Variational Bayesian tracking: Whole track convergence for large-scale ecological video monitoring

“…Locations at corresponding times are joined by black lines and the background picture has been bleached for clarity. the observational noise is robustly modelled by a Student-t distribution and variational Bayes is used for inference [15], [16], [17], [18]. The probability of each observed pixel under this PPCA-AR model p(y t,k | Y T ) is used to quantify whether a pixel belongs to the foreground or to the background.…”

Hierarchical Bayesian ARX models for robust inference

“…As we do not know much about these parameters, vague (non-informative) gamma priors are used as in Christmas and Everson [2011] …”

“…The present work differs from these contributions, mainly in the use of Student's t distributed innovations. Similar models are also considered by Christmas and Everson [2011], who derive a variational Bayes algorithm for the inference problem. This approach is not based on Monte Carlo sampling, but instead makes use of certain deterministic approximations to overcome the intractable integrals that appear in the expression for the posterior distribution.…”