Distributed Learning of Predictive Structures From Multiple Tasks Over Networks

Hua, Junhao; Li, Chunguang; Shen, Hui-Liang

doi:10.1109/tie.2016.2588463

Cited by 16 publications

(8 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the Bayesian learning problem in wireless sensor networks, references [10,13,19] systematically studied how to solve the Bayesian learning problem by using the variational Bayes method in a distributed environment. For the problem of Bayesian inference and estimation on the network, reference [10] proposed a general framework of distributed variational seeing algorithm for conjugate exponential family models.…”

Section: Distributed Bayesian Learningmentioning

confidence: 99%

“…Intelligent nodes are terms such as autonomous agents, which mainly include sensors, processors, actuators, and etc. With the aid of algebraic graph theory, theses nodes are not only communication by network topology, but also allow us to exchange information within their neighborhood and to accomplish network global tasks by online learning [13]. So the IWSN node with distributed manner have "network awareness", which constructed complicated intelligent network systems.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The Algorithms of Distributed Learning and Distributed Estimation about Intelligent Wireless Sensor Network

Tan

2020

Sensors

View full text Add to dashboard Cite

The intelligent wireless sensor network is a distributed network system with high “network awareness”. Each intelligent node (agent) is connected by the topology within the neighborhood which not only can perceive the surrounding environment, but can adjusts its own behavior according to its local perception information to constructs a distributed learning algorithms. Therefore, three basic intelligent network topologies of centralized, non-cooperative, and cooperative are intensively investigated in this paper. The main contributions of the paper include two aspects. First, based on algebraic graph, three basic theoretical frameworks for distributed learning and distributed parameter estimation of cooperative strategy are surveyed: increment strategy, consensus strategy, and diffusion strategy. Second, based on classical adaptive learning algorithm and online updating law, the implementation process of distributed estimation algorithm and the latest research progress of above three distributed strategies are investigated.

show abstract

Section: Distributed Bayesian Learningmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

The Algorithms of Distributed Learning and Distributed Estimation about Intelligent Wireless Sensor Network

Tan

2020

Sensors

View full text Add to dashboard Cite

show abstract

“…In [22], a communication-efficient estimator based on the debiased lasso is presented. Reference [23] learned a shared predictive structure for tasks by extending [12] to a distributed setting. Following the assumptions of MTFL in [4], two communication-efficient subspace pursuit algorithms are provided in [24].…”

Section: Introductionmentioning

confidence: 99%

“…Different from the Jacobian type method, the variables are sequentially optimized for Gauss-Seidel method. In [23], the optimization method that integrates block coordinate descent method (BCD) with the inexact ADMM is utilized for distributed learning. While a BCD for regularized multi-convex optimization is considered in [39].…”

Section: Introductionmentioning

confidence: 99%

Randomized Neural Networks Based Decentralized Multi-Task Learning via Hybrid Multi-Block ADMM

Xiao

Skoglund

2021

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

show abstract

“…In [20], a communication-efficient estimator based on the debiased lasso is presented. Reference [21] learned a shared predictive structure for tasks by extending [12] to a distributed setting. Following the assumptions of MTFL in [4], two communication-efficient subspace pursuit algorithms are provided in [22].…”

Section: Introductionmentioning

confidence: 99%

Decentralized Multi-Task Learning Based on Extreme Learning Machines

Ye¹,

Xiao²,

Skoglund³

2019

Preprint

View full text Add to dashboard Cite

In multi-task learning (MTL), related tasks learn jointly to improve generalization performance. To exploit the high learning speed of extreme learning machines (ELMs), we apply the ELM framework to the MTL problem, where the output weights of ELMs for all the tasks are learned collaboratively. We first present the ELM based MTL problem in the centralized setting, which is solved by the proposed MTL-ELM algorithm. Due to the fact that many data sets of different tasks are geo-distributed, decentralized machine learning is studied. We formulate the decentralized MTL problem based on ELM as majorized multi-block optimization with coupled bi-convex objective functions. To solve the problem, we propose the DMTL-ELM algorithm, which is a hybrid Jacobian and Gauss-Seidel Proximal multi-block alternating direction method of multipliers (ADMM). Further, to reduce the computation load of DMTL-ELM, DMTL-ELM with first-order approximation (FO-DMTL-ELM) is presented. Theoretical analysis shows that the convergence to the stationary point of DMTL-ELM and FO-DMTL-ELM can be guaranteed conditionally. Through simulations, we demonstrate the convergence of proposed MTL-ELM, DMTL-ELM and FO-DMTL-ELM algorithms, and also show that they can outperform existing MTL methods. Moreover by adjusting the dimension of hidden feature space, there exists a trade-off between communication load and learning accuracy for DMTL-ELM.

show abstract

Distributed Learning of Predictive Structures From Multiple Tasks Over Networks

Cited by 16 publications

References 27 publications

The Algorithms of Distributed Learning and Distributed Estimation about Intelligent Wireless Sensor Network

The Algorithms of Distributed Learning and Distributed Estimation about Intelligent Wireless Sensor Network

Randomized Neural Networks Based Decentralized Multi-Task Learning via Hybrid Multi-Block ADMM

Decentralized Multi-Task Learning Based on Extreme Learning Machines

Contact Info

Product

Resources

About