Federated Ensembles: a literature review

Abstract

Federated learning (FL) allows machine learning algorithms to be applied to decentralized data when data sharing is not an option due to privacy concerns. Ensemble-based learning works by training multiple (weak) classifiers whose output is aggregated. Federated ensembles are ensembles applied to a federated setting, where each classifier in the ensemble is trained on one data location. The aim of this review is to provide an overview of the published literature on federated ensembles, their applications, the methods used, the challenges faced, the proposed solutions and their comparative performance. We searched for publications on federated ensembles on five databases (ACM Digital Library, IEEE, arXiv, Google scholar and Scopus) published after 2016. We found 26 articles describing studies either proposing federated ensemble applications or comparing federated ensembles to other federated learning approaches. Federated ensembles were used for a wide varied applications beyond classification. Advocates of federated ensemble mentioned their ability to handle local biases in data. In comparison to federated learning approaches, federated ensembles underperformed in small sample sizes and highly class imbalanced settings. Only 10 articles discussed privacy guarantees or additional privacy preserving techniques. Federated ensembles represent an interesting alternative to federated averaging algorithms that is inherently privacy preserving. They have proved their versatility but remain underutilized.

Federated Bayesian Network Ensembles

“…These diverse types of bias may create problems when using the traditional federated learning approach. An alternative is the use of federated ensembles, ensembles of classifiers, each of which has been trained on the local data of each party in a federated setting [7]. Ensemble based learning works by combining multiple (weak) classifiers which work together to jointly produce classifications using various voting schemes [8].…”

“…Current research into federated ensembles is limited [7]. There is only a small body of current work specifically looking into federated ensembles.…”