Communication-efficient federated learning via knowledge distillation

Abstract

Federated learning is a privacy-preserving machine learning technique to train intelligent models from decentralized data, which enables exploiting private data by communicating local model updates in each iteration of model learning rather than the raw data. However, model updates can be extremely large if they contain numerous parameters, and many rounds of communication are needed for model training. The huge communication cost in federated learning leads to heavy overheads on clients and high environmental burdens. Here, we present a federated learning method named FedKD that is both communication-efficient and effective, based on adaptive mutual knowledge distillation and dynamic gradient compression techniques. FedKD is validated on three different scenarios that need privacy protection, showing that it maximally can reduce 94.89% of communication cost and achieve competitive results with centralized model learning. FedKD provides a potential to efficiently deploy privacy-preserving intelligent systems in many scenarios, such as intelligent healthcare and personalization.

A chemoenzymatic strategy for site-selective functionalization of native peptides and proteins

“…Using PGA global-dimer-H Rd3-CHis variant, we generated >120 g of MK-1092 in 82% purity and 73% isolated yield over just two steps. This two-pot synthesis substantially improved the yield as compared to the direct conjugation used in the initial chemistry route (19,36).…”

“…2 and Table 1), we explored the application and generality of this protecting-group strategy in the synthesis of our two clinical candidates MK-1092 (5a) and MK-5160 (5b) (Fig. 3) (19). These conjugates are insulin dimers stapled at the B29,B29′ positions with either a trans-1,4-cyclohexyldicarboxyamide or a suberoyl amide linker, respectively.…”

“…Modified insulin molecules will advance understanding of how the insulin receptor is activated and the structure-activity relationships of its binders (18) and may ultimately lead to safer insulinbased medicines for glycemic controls (17). Our laboratories have been pursuing diverse human insulin bioconjugates with promising therapeutic potential and improved pharmacodynamic profiles, including insulin receptor partial agonists MK-1092 and MK-5160 (19) or glucose-responsive insulin MK-2640 (20) (Fig. 1C, 5a to 5c).…”

“…1A; pK a values based on (21)]. Our initial conjugation strategy to access insulin analogs was driven primarily by differences in pK a , which afforded only modest site selectivity (19). As a result, a challenging chromatographic purification was required to separate structurally related compounds, leading to moderate yields of the bioconjugates.…”

“…1B) could enable their site-selective conjugation with any molecule of interest (e.g., sugar, polyethylene glycol, peptide), expediting the synthesis of a diverse array of modified insulins 5 (Fig. 1C) (19). To translate this vision into practice, we planned to exploit the specificity displayed by enzymes to prepare derivatives 3.…”