Pooling critical datasets with Federated Learning

Arfat, Yasir; Mittone, Gianluca; Colonnelli, Iacopo; D’Ascenzo, Fabrizio; Esposito, Roberto; Aldinucci, Marco

doi:10.1109/pdp59025.2023.00057

Cited by 3 publications

(1 citation statement)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The process is usually iterative and based on NNs (FedAvg) [4], even if recently methods based on non-NN distributed boosting algorithms have been proposed [24]. These algorithms allow parties to aggregate any kind of model without making assumptions about the kind of model being aggregated or assuming a training procedure based on gradient descent [29].…”

Section: Introductionmentioning

confidence: 99%

Experimenting With Normalization Layers in Federated Learning on Non-IID Scenarios

Casella,

Esposito,

Sciarappa

et al. 2024

IEEE Access

Self Cite

View full text Add to dashboard Cite

Training Deep Learning (DL) models require large, high-quality datasets, often assembled with data from different institutions. Federated Learning (FL) has been emerging as a method for privacypreserving pooling of datasets employing collaborative training from different institutions by iteratively globally aggregating locally trained models. One critical performance challenge of FL is operating on datasets not independently and identically distributed (non-IID) among the federation participants. Even though this fragility cannot be eliminated, it can be debunked by a suitable optimization of two hyperparameters: layer normalization methods and collaboration frequency selection. In this work, we benchmark five different normalization layers for training Neural Networks (NNs), two families of non-IID data skew, and two datasets. Results show that Batch Normalization, widely employed for centralized DL, is not the best choice for FL, whereas Group and Layer Normalization consistently outperform Batch Normalization, with a performance gain of up to about 15 % in the most challenging non-IID scenario. Similarly, frequent model aggregation decreases convergence speed and mode quality.

show abstract

Section: Introductionmentioning

confidence: 99%