Industry-Scale Orchestrated Federated Learning for Drug Discovery

Abstract

To apply federated learning to drug discovery we developed a novel platform in the context of European Innovative Medicines Initiative (IMI) project MELLODDY (grant n°831472), which was comprised of 10 pharmaceutical companies, academic research labs, large industrial companies and startups. The MELLODDY platform was the first industry-scale platform to enable the creation of a global federated model for drug discovery without sharing the confidential data sets of the individual partners. The federated model was trained on the platform by aggregating the gradients of all contributing partners in a cryptographic, secure way following each training iteration. The platform was deployed on an Amazon Web Services (AWS) multi-account architecture running Kubernetes clusters in private subnets. Organisationally, the roles of the different partners were codified as different rights and permissions on the platform and administrated in a decentralized way. The MELLODDY platform generated new scientific discoveries which are described in a companion paper.

MELLODDY: Cross-pharma Federated Learning at Unprecedented Scale Unlocks Benefits in QSAR without Compromising Proprietary Information

“…This indicates that, in practice, the information transfer occurred generally and broadly across a vast spectrum of assays, many of which would not be amenable to cross-compound federation. Notably, a core cross-end point federation scheme can in principle be extended to enable cross-compound federation by mapping common assays to a shared head model . To secure the benefits of cross-end point federation, such cross-compound extension may then best be reserved to a limited set of amenable assays, such as some safety panel assays that happen to be outsourced by multiple pharma partners to common contract research organizations .…”

Introduction to the Special Issue: AI Meets Toxicology

“…Data from chemical companies are subject to privacy and confidentiality concerns, which makes such data difficult to work with. New emerging approaches can leverage company data by using secure multiparty computation, where calculations are performed using encrypted data, or by means of federated learning, where local models are trained in each company and only gradients are exchanged thus keeping underlying data secure as exemplified by the innovative MELLODDY project . Bassani et al described the experience of Roche scientists, who used an alternative method in which local models predicted an unlabeled set, which was then used to teach the federated model, thus exploiting the idea of surrogate data sharing .…”

Prediction of Small-Molecule Developability Using Large-Scale In Silico ADMET Models

“…Last, we evaluated the possible leakage of information from the MELLODDY models into the public data set. As the MELLODDY models were trained using data from multiple pharma companies, 40 it is likely that these have, to some extent, absorbed information that is present in this public data set. Thus, we assumed that the presence of compounds from the public test set in the MELLODDY training data would lead to an evaluation bias pushing the performance of the bPK model toward overoptimistic estimates.…”