Kernel Recursive Least-Squares Tracker for Time-Varying Regression

Abstract

In this paper, we introduce a kernel recursive least-squares (KRLS) algorithm that is able to track nonlinear, time-varying relationships in data. To this purpose, we first derive the standard KRLS equations from a Bayesian perspective (including a sensible approach to pruning) and then take advantage of this framework to incorporate forgetting in a consistent way, thus enabling the algorithm to perform tracking in nonstationary scenarios. The resulting method is the first kernel adaptive filtering algorithm that includes a forgetting factor in a principled and numerically stable manner. In addition to its tracking ability, it has a number of appealing properties. It is online, requires a fixed amount of memory and computation per time step, incorporates regularization in a natural manner and provides confidence intervals along with each prediction. We include experimental results that support the theory as well as illustrate the efficiency of the proposed algorithm.

Spatio-Temporal Learning With the Online Finite and Infinite Echo-State Gaussian Processes

“…These experiments demonstrate that the algorithms produce high accuracies relative to standard kernels (e.g., isotropic and ARD squared exponential kernels) using sliding-windows as well as state-of-the-art online methods including sparse online Gaussian process (SOGP) [4], kernel recursive least squares (KRLS) [8], [9], locally weighted projection regression (LWPR) [10] and kernel least mean squares (KLMS) [11].…”

“…In this paper, we expand upon this body of work and contribute a novel online training method for ESNs through Bayesian online learning, specifically the SOGP [4]. In fact, the SOGP is intimately connected with the KRLS and its more recent variants [8], [9]. KRLS is a kernelised RLS filter with a sparse dictionary.…”

Sequential Likelihood Ratio Test for Cognitive Radios

“…These issues come from the infinite memory of the system through the hyperparameters (5) and can be avoided using a "back-to-the-prior" strategy [33], [18] on the hyperparameter σ…”

Online Kernel Slow Feature Analysis for Temporal Video Segmentation and Tracking

“…Many techniques that try to bound this set have been presented in the literature [18], [19], [20], [21]. For example, [22] propose online kernel algorithms for classification, regression and novelty detection using a stochastic gradient descent algorithm in the Hilbert space defined by the kernel.…”

“…Although alternatives exist [20], [21], [23], [24], we adopt the reduced set expansion of [10] to ensure constant running time. While we describe the main steps in the following, we refer to [10] for details.…”