Online Distributed Learning Over Networks in RKH Spaces Using Random Fourier Features

Abstract

We present a novel diffusion scheme for online kernel-based learning over networks. So far, a major drawback of any online learning algorithm, operating in a reproducing kernel Hilbert space (RKHS), is the need for updating a growing number of parameters as time iterations evolve. Besides complexity, this leads to an increased need of communication resources, in a distributed setting. In contrast, the proposed method approximates the solution as a fixed-size vector (of larger dimension than the input space) using Random Fourier Features. This paves the way to use standard linear combine-then-adapt techniques. To the best of our knowledge, this is the first time that a complete protocol for distributed online learning in RKHS is presented. Conditions for asymptotic convergence and boundness of the networkwise regret are also provided. The simulated tests illustrate the performance of the proposed scheme.

Adaptive Graph Filters in Reproducing Kernel Hilbert Spaces: Design and Performance Analysis

“…Results show that increasing D reduces the gap between the numerical and theoretical results for the steady-state behavior of the algorithm. This observation is in line with the result presented in [47].…”

“…Hence, z k,n may not be normally distributed. If the basis of the RFF space is generated in a way such that the basis vectors v i = v j for any i = j, the autocorrelation matrix R z,k , for k ∈ N will be strictly positive definite [47].…”

Distributed Adaptive Learning With Multiple Kernels in Diffusion Networks

“…The functional adapt-then-combine KLMS (FATC-KLMS) proposed in [35] is a kernelized version of the algorithm derived in [9]. The random Fourier features diffusion KLMS (RFF-DKLMS) proposed in [36] uses random Fourier features to achieve a fixed-size coefficient vector and to avoid an a priori design of a dictionary set. However, the achievable performance strongly depends on the number of utilized Fourier features.…”

Accelerate Convergence of Polarized Random Fourier Feature-Based Kernel Adaptive Filtering With Variable Forgetting Factor and Step Size

“…As the development of kernel approximation technology, the random Fourier feature was introduced into adaptive signal processing field and a random Fourier features-based kernel least-mean-square (RFFKLMS) algorithm was proposed [16], [17], [18]. The main idea of the random Fourier feature method is to map input data to a relatively lowdimensional feature space by using the approximate explicit mapping function [19], [20].…”

“…Although PRFFKLMS algorithm is the most representative RFF-based KLMS algorithm at present, the convergence speed and tracking speed of PRFFKLMS algorithm still need to be improved for nonstationary signal processing cases. In [18], [21], the RFFbased kernel adaptive filtering algorithm such as RFFKLMS and PRFFKLMS algorithm use the fixed step size strategy.…”