Distributed Adaptive Learning With Multiple Kernels in Diffusion Networks

Shin, Ban-Sok; Yukawa, Masahiro; Cavalcante, Renato L. G.; Dekorsy, Armin

doi:10.1109/tsp.2018.2868040

Cited by 28 publications

(15 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A privacy-preserving distributed learning is witnessing an unprecedented interest, in which local data is kept at distributed edge devices without centralizing the data. According to the structures of a communication network, this can be categorized into a centralized federated learning [6], [7], [8], [9] and a fully distributed learning (a.k.a., a fully decentralized federated learning) [10], [11], [12], [13].…”

Section: Introductionmentioning

confidence: 99%

“…A kernel-based learning has been extended into a fully decentralized network [10], [11], [12] due to its necessity in various applications such as social networks, big data processing, and environmental monitoring. In [10], a consensus-based distributed MKL has been proposed for regression tasks, where alternating direction method of multipliers (ADMM) [19] is used as the underlying distributed optimization technique.…”

Section: Introductionmentioning

confidence: 99%

“…A diffusion-based distributed learning with a single kernel has been developed in [11], where the cooperation of distributed nodes is performed via the diffusion rationale. In addition, such diffusion-based approach has been incorporated into MKL framework [12], being able to outperform the previous works [10], [11] in distributed estimation tasks. However, none of the above kernel-based approaches cannot operate in distribute online learning frameworks.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Distributed Online Learning with Multiple Kernels

Chae¹,

Hong²

2021

Preprint

View full text Add to dashboard Cite

We consider the problem of learning a nonlinear function over a network of learners in a fully decentralized fashion. Online learning is additionally assumed, where every learner receives continuous streaming data locally. This learning model is called a fully distributed online learning (or a fully decentralized online federated learning). For this model, we propose a novel learning framework with multiple kernels, which is named DOMKL. The proposed DOMKL is devised by harnessing the principles of an online alternating direction method of multipliers and a distributed Hedge algorithm. We theoretically prove that DOMKL over T time slots can achieve an optimal sublinear regret O( √ T ), implying that every learner in the network can learn a common function which has a diminishing gap from the best function in hindsight. Our analysis also reveals that DOMKL yields the same asymptotic performance of the stateof-the-art centralized approach while keeping local data at edge learners. Via numerical tests with real datasets, we demonstrate the effectiveness of the proposed DOMKL on various online regression and time-series prediction tasks.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Distributed Online Learning with Multiple Kernels

Chae¹,

Hong²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…By transforming the input data to the high dimensional feature space, nonlinear problems can be modeled linearly. The kernel-based least squares (LS) estimation [3] and varieties of kernel adaptive filters [4], [5] have been studied for nonlinear estimation. However, when estimating dynamic fields, LSbased methods may have limited performance.…”

Section: Introductionmentioning

confidence: 99%

Diffusion Field Estimation Using Decentralized Kernel Kalman Filter with Parameter Learning Over Hierarchical Sensor Networks

Wang

Shin

Shutin

et al. 2020

2020 IEEE 30th International Workshop on Machine Learning for Signal Processing (MLSP)

Self Cite

View full text Add to dashboard Cite

In this paper, a task is addressed to track a nonlinear timevarying diffusion field based on data collected by sensor networks. By exploiting kernel methods, the nonlinear field function is approximated by a linear combination of kernel functions in a reproducing kernel Hilbert space (RKHS). To capture the dynamical property of a diffusion field and the relation of system input and output data, a state-space model on weights of these kernel functions is constructed with unknown process noise. Thus, the nonlinear tracking problem is transformed into a linear state estimation solved by Kalman filter. Further, this kernel Kalman filter (KKF) is decomposed into a decentralized fashion in a way to collect sensor data efficiently over a hierarchical network structure with different clusters. To adapt the algorithm to unknown process noise, a decentralized variational Bayesian KKF is proposed to learn the distributions of system unknown variables.

show abstract

“…For efficient online learning, different methods for adaptively adjusting kernel parameters are introduced, which can change these parameters at different filtering stages, efficiently [25]. Another effective kernel selection strategy is the multikernel learning based method [26]- [28], which can combine several distinct kernels to achieve desirable performance. It is worth noting that the multikernel method can use different types of kernels rather than the same type of kernels with different kernel parameters.…”

Section: Introductionmentioning

confidence: 99%

Multikernel Adaptive Filters Under the Minimum Cauchy Kernel Loss Criterion

2019

View full text Add to dashboard Cite

The Cauchy loss has been successfully applied in robust learning algorithms in the presence of large outliers, but it may suffer from performance degradation in complex nonlinear tasks. To address this issue, by transforming the original data into the reproducing kernel Hilbert spaces (RKHS) with the kernel trick, a novel Cauchy kernel loss is developed in such a kernel space. Based on the minimum Cauchy kernel loss criterion, the multikernel minimum Cauchy kernel loss (MKMCKL) algorithm is proposed by mapping the input data into the multiple RKHS. The proposed MKMCKL algorithm can provide the performance improvement of the kernel adaptive filter (KAF) based on a single kernel, and also improve the stability of the multikernel adaptive filter based on the quadratic loss in impulsive noises, efficiently. To further curb the growth of network of MKMCKL, a novel sparsification method is presented to prune redundant data, thus reducing its computational and storage burdens. Simulations on different nonlinear applications illustrate the performance superiorities of the proposed algorithms in impulsive noises.

show abstract

Distributed Adaptive Learning With Multiple Kernels in Diffusion Networks

Cited by 28 publications

References 51 publications

Distributed Online Learning with Multiple Kernels

Distributed Online Learning with Multiple Kernels

Diffusion Field Estimation Using Decentralized Kernel Kalman Filter with Parameter Learning Over Hierarchical Sensor Networks

Multikernel Adaptive Filters Under the Minimum Cauchy Kernel Loss Criterion

Contact Info

Product

Resources

About