Adaptive multitask network based on maximum correntropy learning algorithm

Abstract

Adaptive networks solve distributed optimization problems in which all agents of the network are interested to collaborate with their neighbors to learn a similar task. Collaboration is useful when all agents seek a similar task. However, in many applications, agents may belong to different clusters that seek dissimilar tasks. In this case, nonselective collaboration will lead to damaging results that are worse than noncooperative solution. In this paper, we contribute in problems that several clusters of interconnected agents are interested in learning multiple tasks. To address multitask learning problem, we consider an information theoretic criterion called correntropy in a distributed manner providing a novel adaptive combination policy that allows agents to learn which neighbors they should cooperate with and which other neighbors they should reject. In doing so, the proposed algorithm enables agents to recognize their clusters and to achieve improved learning performance compared with noncooperative strategy. Stability analysis in the mean sense and also a closed-form relation determining the network error performance in steady-state mean-square-deviation is derived. Simulation results illustrate the theoretical findings and match well with theory. KEYWORDSadaptive network, correntropy criterion, distributed processing, multitask learning 1 1232

Transient performance analysis of adaptive multitask network based on correntropy criterion

“…The interpretation of this algorithm is clarified in our previous work 18 and a brief summary is presented in Table 1. Solving multitask learning problems based on this approach is useful since cooperation between sensors with different tasks is totally limited via the Gaussian kernel of the local error.…”

“…Therefore, for sufficiently large i, ie, (i > T s ), the neighborhood set can be assumed to be independent of time, and asymptotically, we have the following 18 : Therefore, for sufficiently large i, ie, (i > T s ), the neighborhood set can be assumed to be independent of time, and asymptotically, we have the following 18 :…”

“…[19][20][21][22][23][24][25][26][27][28][29][30][31] The proposed learning algorithm determines a novel cooperation policy for sensors, in which they can collaborate with neighbors that have similar task and they can reject the collaboration of neighbors that have distinct tasks. In this paper, we want to analyze the transient performance of our proposed learning algorithm 17,18 and compare the theoretical results with simulation results. This paper is organized as follows.…”

“…Other significant works conducting multitask problems are introduced in previous works. [11][12][13][14][15][16][17][18] The clustering method proposed in the work of Zhao and Sayed 12 and Chen et al 13 is based on solving a hypothesis test for determining the new neighborhood. Another method to exploit similarities between tasks is to solve optimization problems with the regularization technique.…”

Generalized maximum correntropy detector for non‐Gaussian environments

“…Inspired by ITL, a generalized similarity measure called correntropy was developed. 22,23 The cross-correntropy for 2 arbitrary scalar random variables X and Y, is formally defined as…”