Addressing the clinical unmet needs in primary Sjögren’s Syndrome through the sharing, harmonization and federated analysis of 21 European cohorts

FHBF: Federated hybrid boosted forests with dropout rates for supervised learning tasks across highly imbalanced clinical datasets

“…To evaluate the performance of the FHBF against the existing high-performance federated learning algorithms (i.e., FGBT and FDART 14 ), we gained access to a pan-European data hub on rare autoimmune diseases, which includes 21 databases as part of the HarmonicSS EU Project. 31 The patient data were shared, curated, harmonized, 14 and stored in private spaces within a cloud environment. Socio-demographic information was present for 6,060 patient records that were included in the analysis.…”

“… 2 Online learning methods, on the other hand, are restricted to the additive update of the weights of the ML model on new “online” training instances. A solution to this is to use incremental learning, 11 , 12 , 13 , 14 which trains a classifier on an initial database and then incrementally adjusts the weights of the classifier on a series of existing databases. Toward this direction, many incremental learning algorithms have been proposed, including the family of the multiple additive regression trees (MART), 14 , 15 , 16 the support vector machines (SVM), 17 , 18 and the multinomial naive Bayes, 14 where the gradient boosting trees (GBT) algorithm is the most popular implementation of MART with favorable performance in diverse classification tasks.…”

“…Toward this direction, many incremental learning algorithms have been proposed, including the family of the multiple additive regression trees (MART), 14 , 15 , 16 the support vector machines (SVM), 17 , 18 and the multinomial naive Bayes, 14 where the gradient boosting trees (GBT) algorithm is the most popular implementation of MART with favorable performance in diverse classification tasks. 14 , 15 …”

“…A common problem with the FGBT algorithm, however, lies in the fact that the trees that are added early in the ensemble, at a particular stage, tend to have a higher impact during the decision-making process than those added later. 14 , 20 To this end, dropouts have been recently adopted by the deep learning community 19 , 20 to deal with this issue, by scaling the most prominent trees in the ensemble, with a specific rate of rejected trees. On the other hand, a main problem in FGBT with dropouts is to account for overfitting effects in the selection of the dropout rate which is arbitrary.…”

“…On the other hand, naive Bayes approaches, like the multinomial naive Bayes, are partially affected by overfitting, are often less flexible since they assume feature independence. In addition, although the GBT (with and without dropouts) algorithm has been widely used in the literature as a state-of-the art classifier, with advanced implementations both in centralized and federated environments, 14 , 15 , 16 , 19 none of these existing studies have investigated the loss during the training and testing across multiple and highly imbalanced datasets within federated environments.…”