End-to-End deep structure generative model for protein design

Abstract

Designing protein with desirable structure and functional properties is the pinnacle of computational protein design with unlimited potentials in the scientific community from therapeutic development to combating the global climate crisis. However, designing protein macromolecules at scale remains challenging due to hard-to-realize structures and low sequence design success rate. Recently, many generative models are proposed for protein design but they come with many limitations. Here, we present a VAE-based universal protein structure generative model that can model proteins in a large fold space and generate high-quality realistic 3-dimensional protein structures. We illustrate how our model can enable robust and efficient protein design pipelines with generated conformational decoys that bridge the gap in designing structure conforming sequences. Specifically, sequences generated from our design pipeline outperform native fixed backbone design in 856 out of the 1,016 tested targets(84.3%) through AF2 validation. We also demonstrate our model's design capability and structural pre-training potential by structurally inpainting the complementarity-determining regions(CDRs) in a set of monoclonal antibodies and achieving superior performance compared to existing methods.

DiffSDS: A language diffusion model for protein backbone inpainting under geometric conditions and constraints

“…Unconditional protein structure generation Anand's model (Anand & Huang, 2018) Noise PyTorch RamaNet (Sabban & Markovsky, 2020) Noise TF Ig-VAE (Eguchi et al, 2022) Noise PyTorch FoldingDiff (Wu et al, 2022a) Noise PyTorch Protein seqeunce design GraphTrans (Ingraham et al, 2019) 3D Backbone PyTorch GVP (Jing et al, 2020) 3D Backbone PyTorch GCA (Tan et al, 2022) 3D Backbone PyTorch AlphaDesign (Gao et al, 2022a) 3D Backbone PyTorch ESM-IF (Hsu et al, 2022) 3D Backbone PyTorch ProteinMPNN (Dauparas et al, 2022) 3D Backbone PyTorch PiFold (Gao et al, 2022b) 3D Backbone PyTorch Conditional protein design ProteinSGM (Lee & Kim, 2022) Masked structures -Wang's model (Wang et al, 2022a) Functional sites PyTorch SMCDiff (Trippe et al, 2022) Functional motifs -CoordVAE (Lai et al, 2022) Backbone Template -CEM (Fu & Sun, 2022) CDR geometry -Tischer's model (Tischer et al, 2020) Functional motifs TF Anand's model (Anand & Achim, 2022) Multiple conditions -RefineGNN (Jin et al, 2021) Antigen structure PyTorch DiffAb (Luo et al) Antigen structure PyTorch Forward process We start from the standard diffusion process x 0 → x 1 → • • • → x T , where the forward translation kernel from timestamp s to t is defined as q(x t |x s ) = N (x t ; α t|s x s , σ 2 t|s I), s ≤ t. Denote α t = α t|0 , σ t = σ t|0 , and q(x 0 |x 0 ) = N (x 0 ; α 0 x, σ 2 0 I), α 0 = 1, σ 0 = 0. We will show that α t|s = α t /α s , σ 2 t|s = σ 2 t − α 2 t|s σ 2 s .…”

“…Protein Design In addition to small molecules, biomolecules such as proteins have also attracted considerable attention by researchers (Ding et al, 2022;Ovchinnikov & Huang, 2021;Gao et al, 2020;Strokach & Kim, 2022). We divide the mainstream protein design methods into three categories: protein sequence design (Li et al, 2014;Wu et al, 2021;Pearce & Zhang, 2021;Ingraham et al, 2019;Jing et al, 2020;Tan et al, 2022;Gao et al, 2022a;Hsu et al, 2022;Dauparas et al, 2022;Gao et al, 2022b;O'Connell et al, 2018;Wang et al, 2018;Qi & Zhang, 2020;Strokach et al, 2020;Chen et al, 2019;Zhang et al, 2020;Anand & Achim, 2022), unconditional protein structure generation (Anand & Huang, 2018;Sabban & Markovsky, 2020;Eguchi et al, 2022;Wu et al, 2022a), and conditional protein design (Lee & Kim, 2022;Wang et al, 2022a;Trippe et al, 2022;Lai et al, 2022;Fu & Sun, 2022;Tischer et al, 2020;Anand & Achim, 2022;. Protein sequence design aims to discover protein sequences folding into the desired structure, and unconditional protein structure generation focus on generating new protein structures from noisy inputs.…”

“…ProteinSGM (Lee & Kim, 2022) mask short spans (< 8 residues) of different secondary structures in different structures and treats the design task as a inpainting problem. CoordVAE (Lai et al, 2022) produces novel protein structures conditioned on the backbone template. RefineGNN (Jin et al, 2021), CEM (Fu & Sun, 2022), and DiffAb (Luo et al) aim to generate the complementarity-determining regions of the antibody.…”

TopoDiff: Improving Protein Backbone Generation with Topology-aware Latent Encoding

“…People have used Generative Adversarial Networks (GAN [10]) [11] and Variational Autoencoders (VAE [12]) [13]–[17] to model the high-dimensional structural space. Those models typically represent the proteins as 2D distance matrices and the outputs were often a predicted distance matrix that frequently suffered from inconsistency issues [18].…”

From sequence to function through structure: deep learning for protein design

“…IG-VAE [74] addressed some of these shortcomings by training a variational autoencoder (VAE) that directly generated 3D coordinates of backbone atoms for class-specific Immunoglobulin proteins. The VAE implemented in Lai [75] instead outputs a conformational ensemble of protein structures. Similar methods with the goal of generating structures are RamaNet [76], a long-short term memory network (LSTM), trained in an autoregressive manner to output a sequence of φ and Ψ angles to design alpha-helical structures, and DECO-VAE [77], based on VAEs.…”