Most teacher-student frameworks based on knowledge distillation (KD) depend on a strong congruent constraint on instance level. However, they usually ignore the correlation between multiple instances, which is also valuable for knowledge transfer. In this work, we propose a new framework named correlation congruence for knowledge distillation (CCKD), which transfers not only the instance-level information, but also the correlation between instances. Furthermore, a generalized kernel method based on Taylor series expansion is proposed to better capture the correlation between instances. Empirical experiments and ablation studies on image classification tasks (including CIFAR-100, ImageNet-1K) and metric learning tasks (including ReID and Face Recognition) show that the proposed CCKD substantially outperforms the original KD and achieves stateof-the-art accuracy compared with other SOTA KD-based methods. The CCKD can be easily deployed in the majority of the teacher-student framework such as KD and hintbased learning methods. Our code will be released, hoping to nourish our idea to other domains.

Glycosylation reactions mainly catalyzed by glycosyltransferases (Gts) occur almost everywhere in the biosphere, and always play crucial roles in vital processes. In order to understand the full potential of Gts, the chemical and structural glycosylation mechanisms are systematically summarized in this review, including some new outlooks in inverting/retaining mechanisms and the overview of GT-C superfamily proteins as a novel Gt fold. Some special features of glycosylation and the evolutionary studies on Gts are also discussed to help us better understand the function and application potential of Gts. Natural product (NP) glycosylation and related Gts which play important roles in new drug development are emphasized in this paper. The recent advances in the glycosylation pattern (particularly the rare C- and S-glycosylation), reversibility, iterative catalysis and protein auxiliary of NP Gts are all summed up comprehensively. This review also presents the application of NP Gts and associated studies on synthetic biology, which may further broaden the mind and bring wider application prospects.

No abstract

No abstract

Distillation-based learning boost the performance of the miniaturized neural network based on the hypothesis that the representation of a teacher model can be used as structured and relatively weak supervision, and thus would be easily learned by a miniaturized model. However, we find that the representation of a converged heavy model is still a strong constraint for training a small student model, which leads to a high lower bound of congruence loss. In this work, inspired by [1] we consider the knowledge distillation from the perspective of curriculum learning by routing. Instead of supervising the student model with a converged teacher model, we supervised it with some anchor points selected from the route in parameter space that the teacher model passed by, as we called route constrained optimization (RCO). We experimentally demonstrate this simple operation greatly reduces the lower bound of congruence loss for knowledge distillation, hint and mimicking learning. On close-set classification tasks like CIFAR [16] and ImageNet [3], RCO improves knowledge distillation by 2.14% and 1.5% respectively. For the sake of evaluating the generalization, we also test RCO on the open-set face recognition task MegaFace. RCO achieves 84.3% accuracy on 1 vs. 1 million task with only 0.8 M parameters, which push the SOTA by a large margin.

No abstract

Nisin, one kind of natural antimicrobial peptide, is produced by certain Lactococcus lactis strains, which generally require expensive high-quality nitrogen sources due to limited ability of amino acids biosynthesis. Here we use defatted soybean meal (DSM) as sole nitrogen source to support L. lactis growth and nisin production. DSM medium composition and fermentation conditions were optimized using the methods of Plackett-Burman design and central composite design. The highest nisin production of 3879.58 IU/ml was obtained in DSM medium, which was 21.3% higher than that of commercial medium. To further increase the utilization ability of nitrogen sources, we enhanced the proteolytic function in L. lactis through rationally expressing the related enzymes, which were selected according to the compositions of amino acids and molecular weight of peptides in DSM medium. Significantly, an artificial proteolytic system consisting of a heterologous protease (NprB), an oligopeptides transporter subunit (OppA) and two peptidases (PepF and PepM) was introduced into L.lactis. The constructed strain BAFM was capable of achieving efficient biomass accumulation and nisin yield with 30% decreased amount of DSM hydrolysates, which further reduced the cost of nisin production. The strategy described here offers opportunities for low-cost L. lactis fermentation and large-scale nisin production in industry.

No abstract