Distributed on-line learning for random-weight fuzzy neural networks

IEEE Trans. Fuzzy Syst.

Chang

et al. 2023

Distributed fuzzy neural networks (DFNNs) have attracted increasing attention recently due to their learning abilities in handling data uncertainties in distributed scenarios. However, it is challenging for DFNNs to handle cases in which the local data are non-independent and identically distributed (non-IID). In this paper, we propose a federated fuzzy neural network (FedFNN) with evolutionary rule learning (ERL) to cope with non-IID issues as well as data uncertainties. The FedFNN maintains a global set of rules in a server and a personalized subset of these rules for each local client. ERL is inspired by the theory of biological evolution; it encourages rule variations while activating superior rules and deactivating inferior rules for local clients with non-IID data. Specifically, ERL consists of two stages in an iterative procedure: a rule cooperation stage that updates global rules by aggregating local rules based on their activation statuses and a rule evolution stage that evolves the global rules and updates the activation statuses of the local rules. This procedure improves both the generalization and personalization of the FedFNN for dealing with non-IID issues and data uncertainties. Extensive experiments conducted on a range of datasets demonstrate the superiority of the FedFNN over state-of-the-art methods. Our code is available online 1

Section: Related Work a Distributed Fuzzy Neural Networkmentioning

confidence: 99%

Federated Fuzzy Neural Network With Evolutionary Rule Learning

Zhang

IEEE Trans. Fuzzy Syst.

Chang

et al. 2023

“…For example, Fierimonte et al [44] developed a decentralized FNN with random weights, where parameters in the fuzzy membership functions are chosen randomly as opposed to being trained. In subsequent work, they introduced an online implementation of the same FNN structure [45]. Notably, a random method of identifying parameters can result in very large deviations in accuracy during the learning process.…”

Section: A Distributed Learningmentioning

confidence: 99%

“…Notably, a random method of identifying parameters can result in very large deviations in accuracy during the learning process. Additionally, the algorithms are only applied in the consequent layers of the FNN [44], [45], which means that, strictly speaking, this decentralized FNN model is only partially distributed. A more recent proposition by Shi et al [46] involves a distributed FNN with a consensus learning strategy.…”

Section: A Distributed Learningmentioning

confidence: 99%

Distributed Semisupervised Fuzzy Regression With Interpolation Consistency Regularization

IEEE Trans. Fuzzy Syst.

Zhang

Cao

et al. 2022

Recently, distributed semi-supervised learning (DSSL) algorithms have shown their effectiveness in leveraging unlabeled samples over interconnected networks, where agents cannot share their original data with each other and can only communicate non-sensitive information with their neighbors. However, existing DSSL algorithms cannot cope with data uncertainties and may suffer from high computation and communication overhead problems. To handle these issues, we propose a distributed semi-supervised fuzzy regression (DSFR) model with fuzzy if-then rules and interpolation consistency regularization (ICR). The ICR, which was proposed recently for semi-supervised problem, can force decision boundaries to pass through sparse data areas, thus increasing model robustness. However, its application in distributed scenarios has not been considered yet. In this work, we proposed a distributed Fuzzy C-means (DFCM) method and a distributed interpolation consistency regularization (DICR) built on the well-known alternating direction method of multipliers to respectively locate parameters in antecedent and consequent components of DSFR. Notably, the DSFR model converges very fast since it does not involve back-propagation procedure and is scalable to large-scale datasets benefiting from the utilization of DFCM and DICR. Experiments results on both artificial and real-world datasets show that the proposed DSFR model can achieve much better performance than the state-of-the-art DSSL algorithm in terms of both loss value and computational cost. Our code is available online 1 .

“…Recently, several distributed algorithms for FNN were proposed [23], [24]. The authors in [23] proposed a decentralized algorithm for random-weights FNN, where the parameters in the antecedent layer are randomly selected instead of being estimated.…”

mentioning

confidence: 99%

“…The authors in [23] proposed a decentralized algorithm for random-weights FNN, where the parameters in the antecedent layer are randomly selected instead of being estimated. An online implementation for the same FNN structure in [23] is further proposed in [24]. There's no doubt that such a random method for parameter identification can result in very large deviations during the learning process.…”

mentioning

confidence: 99%

Consensus Learning for Distributed Fuzzy Neural Network in Big Data Environment

IEEE Trans. Emerg. Top. Comput. Intell.

Lin

Chang

et al. 2021

Uncertainty and distributed nature inherently exist in big data environment. Distributed fuzzy neural network (D-FNN) that not only employs fuzzy logics to alleviate the uncertainty problem but also deal with data in a distributed manner, is effective and crucial for big data. Existing D-FNNs always avoided consensus for their antecedent layer due to computational difficulty. Hence such D-FNNs are not really distributed since a single model can not be agreed by multiple agents. This paper proposes a real D-FNN model to handle the uncertainty and distributed challenges in the big data environment. The proposed D-FNN model considers consensus for both the antecedent and consequent layers. A novel consensus learning, which involves a distributed structure learning and a distributed parameter learning, is proposed to handle the D-FNN model. The proposed consensus learning algorithm is built on the well-known alternating direction method of multipliers, which does not exchange local data among agents. The major contribution of this paper is to propose the real D-FNN model for the big data and the novel consensus learning algorithm for this D-FNN model. Simulation results on widespread datasets demonstrate the superiority and effectiveness of the proposed D-FNN model and consensus learning algorithm.