Decentralized Sparse Multitask RLS Over Networks

Cao, Xuanyu; Liu, K. J. Ray

doi:10.1109/tsp.2017.2750110

Cited by 19 publications

(8 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The computation of the stepsize α n in (45) requires information from all entries of tr i,n , p i,n u, and is used in turn to update all their entries in (46) and (47). This can be motivated from the fact that P ´1 i reflects correlation, either among the individual entries of u i , or through Π itself.…”

Section: B Decoupled Recursionsmentioning

confidence: 99%

Optimal Diffusion Learning Over Networks—Part II: Multitask Algorithms

Merched

2022

IEEE Open J. Signal Process.

View full text Add to dashboard Cite

Section: B Decoupled Recursionsmentioning

confidence: 99%

Optimal Diffusion Learning Over Networks—Part II: Multitask Algorithms

Merched

2022

IEEE Open J. Signal Process.

View full text Add to dashboard Cite

“…It turns out that cooperation significantly reduces the MSD and the residual bias in their proposed scheme. More recently, distributed sparse RLS adaptation schemes in the single-task and multitask settings were indroduced in [20] and [21]. The RLS-diffusion scheme was extended in [22], where it is shown theoretically and via simulations that the resulting algorithm provides a good trade-off between estimation and the cost of internode communication.…”

Section: B Prior Workmentioning

confidence: 99%

“…" " " W i . The multitask mode is characterized by arbitrary choices of η, where agents estimate their own local parameters under the smoothness condition (29). Note that setting a large η Ñ 8 suggests that we can compute the linearly constrained solution almost exactly as W c i " W 8,i 2 .…”

Section: Multitask Formulationmentioning

confidence: 99%

Optimal Diffusion Learning Over Networks—Part I: Single-Task Algorithms

Merched

2022

IEEE Open J. Signal Process.

View full text Add to dashboard Cite

We revisit the theory of distributed networks of cooperative agents under a broader perspective of diffusion adaptation, by exploiting proximity concepts. This leads to two main families of algorithms with enhanced convergence rate and mean-square-error performance. Part I of this work considers mainly single-task scenarios, which are based on formulating optimal learning and fusion steps via an adaptive network penalty function. The main recursions, which we refer to as Adapt-and-Fuse (AAF) diffusion, are reminiscent of a reweighted network regularized algorithm, usually seen in standalone formulations. This is in line with early approaches that promote proximity among agents in cooperative networks. The AAF strategy employs exact fusion in the leastsquares sense, and outperforms the exact global least-squares solution that ignores the topology of the network. It also suggests simplified LMS-complexity algorithms, and motivates us to develop a normalized version of the relative variance diffusion algorithm, which also learns combination weights. It is verified that even when agents do not share estimates, but only their uncertainties, the simplified AAF improves accuracy over the NLMS-RV algorithm in the presence of intruders, and becomes more robust to noisy links. In order to cope with the computational burden associated with long parameter vectors and correlated inputs, an overlapped block multidelay adaptive frequency-domain (FD) version of each new algorithm is derived. It turns out that for correlated inputs, these FD-LMS versions outperform the exact fullband RLS solutions. In the accompanying Part II of this work, we pursue extensions to the multitask scenario. Extensive simulations illustrate the superiority of the new approaches.

show abstract

“…References [13]- [15] provide variations for such problems for the special case of mean-square-error costs. Here we treat general convex costs.…”

Section: Assumption 1 (Strong Convexity)mentioning

confidence: 99%

Learning Over Multitask Graphs—Part I: Stability Analysis

Nassif

Vlaski

Richard

et al. 2020

IEEE Open J. Signal Process.

View full text Add to dashboard Cite

This paper formulates a multitask optimization problem where agents in the network have individual objectives to meet, or individual parameter vectors to estimate, subject to a smoothness condition over the graph. The smoothness condition softens the transition in the tasks among adjacent nodes and allows incorporating information about the graph structure into the solution of the inference problem. A diffusion strategy is devised that responds to streaming data and employs stochastic approximations in place of actual gradient vectors, which are generally unavailable.The approach relies on minimizing a global cost consisting of the aggregate sum of individual costs regularized by a term that promotes smoothness. We show in this Part I of the work, under conditions on the step-size parameter, that the adaptive strategy induces a contraction mapping and leads to small estimation errors on the order of the small step-size. The results in the accompanying Part II will reveal explicitly the influence of the network topology and the regularization strength on the network performance and will provide insights into the design of effective multitask strategies for distributed inference over networks.

show abstract

Decentralized Sparse Multitask RLS Over Networks

Cited by 19 publications

References 50 publications

Optimal Diffusion Learning Over Networks—Part II: Multitask Algorithms

Optimal Diffusion Learning Over Networks—Part II: Multitask Algorithms

Optimal Diffusion Learning Over Networks—Part I: Single-Task Algorithms

Learning Over Multitask Graphs—Part I: Stability Analysis

Contact Info

Product

Resources

About