A Unifying View of Wiener and Volterra Theory and Polynomial Kernel Regression

Abstract

Volterra and Wiener series are perhaps the best-understood nonlinear system representations in signal processing. Although both approaches have enjoyed a certain popularity in the past, their application has been limited to rather low-dimensional and weakly nonlinear systems due to the exponential growth of the number of terms that have to be estimated. We show that Volterra and Wiener series can be represented implicitly as elements of a reproducing kernel Hilbert space by using polynomial kernels. The estimation complexity of the implicit representation is linear in the input dimensionality and independent of the degree of nonlinearity. Experiments show performance advantages in terms of convergence, interpretability, and system sizes that can be handled.

Kernel methods in system identification, machine learning and function estimation: A survey

“…Other regularized approaches for nonlinear and state-space models identification can be found in Frigola, Lindsten, Schon, and Rasmussen (2013), and Hall, Rasmussen, and Maciejowski (2012). A connection between Volterra and Wiener nonlinear system representation and the RKHS induced by the polynomial kernel, together with an efficient identification scheme, can be found in Franz and Schölkopf (2006). Other recent applications regard Wiener, Hammerstein and Wiener-Hammerstein system identification (Falck et al, 2012;Falck, Pelckmans, Suykens, & De Moor, 2009;Goethals, Pelckmans, Falck, Suykens, & De Moor, 2010;Goethals, Pelckmans, Suykens, & De Moor, 2005;Lindsten, Schön, & Jordan, 2013).…”

Modeling nonlinear circuits with linearized dynamical models via kernel regression

“…Now we will introduce a map y (·) : R N ! Y which forms an injection into a high dimensional space Y where the nonlinear function g(·) can be replaced by the linear operator G , and the nonlinear function h(·) can be replaced by the linear operator H [5].…”

“…This linearization of the dynamical system is dependent on our ability to find a set of maps y (·), x d (·) under which the nonlinear functions g(·), h(·) can be represented by the linear operators G, H. While this concept has solid theoretical support and has been developed for certain categories of kernels [5], the selection of the map and its associated kernel function is potentially very difficult and its existence is not guaranteed.…”

Does Cognitive Science Need Kernels?

“…Here, we have mainly considered radial basis functions because of their psychological and neural plausibility. However, polynomial kernels have some plausibility, too (Jäkel et al, 2007) and they can potentially be used for identifying critical features via Wiener and Volterra Theory (Franz & Schölkopf, 2006). There are also kernels that can deal with non-vectorial stimuli, like strings, trees or graphs (Hofmann et al, 2008).…”