Kalman Filter With Impulse Noised Outliers : a Robustsequential Algorithm to Filter Data With a Large Number Of outliers

Abstract

Impulse noised outliers are data points that differs significantly from other observations. They are generally removed from the data set through local regression or Kalman filter algorithm. However, these methods, or their generalizations, are not well suited when the number of outliers is of the same order as the number of low-noise data. In this article, we propose a new model for impulse noised outliers based on simple latent linear Gaussian processes as in the Kalman Filter. We present a fast forward-backward algorithm to filter and smooth sequential data and which also detect these outliers. We compare the robustness and efficiency of this algorithm with classical methods. Finally, we apply this method on a real data set from a Walk Over Weighing system admitting around $60\%$ of outliers. For this application, we further develop an (explicit) EM algorithm to calibrate some algorithm parameters.