Protein Structure and Sequence Generation with Equivariant Denoising Diffusion Probabilistic Models

Anand, Namrata; Achim, Tudor

doi:10.48550/arxiv.2205.15019

Cited by 59 publications

(81 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Following previous works (Jin et al, 2021;Jing et al, 2021;Anand & Achim, 2022), we conduct extensive experiments and evaluate the proposed PROTSEED on the following three tasks: Antibody CDR Co-Design (Section 4.1), Protein Sequence-Structure Co-Design (Section 4.2), and Fixed Backbone Sequence Design (Section 4.3). We also show cases where PROTSEED successfully conducts de novo protein sequence design with new folds in Section 4.4.…”

Section: Methodsmentioning

confidence: 99%

“…Note that context features vary from setting to setting. For example, in antibody CDR design (Jin et al, 2021;Luo et al, 2022), they are derived from antibody framework and binding antigen structures with CDR region masked, while in full protein design (Anand & Achim, 2022), they can be secondary structure annotations and residue-residue contact features.…”

Section: Preliminariesmentioning

confidence: 99%

“…We then adopt a variant of Invariant Point Attention (IPA) (Jumper et al, 2021) called SeqIPA to capture the interplay of residue types, residue structures and context features, integrating them all altogether into updated context features. Such a practice is often favored in literature (Anand & Achim, 2022;Luo et al, 2022;Tubiana et al, 2022) as it is aware of the orientation of each residue frame while being roto-translation invariant to input and output features. Distinct from vanilla IPA, our SeqIPA takes residue types as the additional input to bias the attention map and steer the representation of the whole protein generated so far:…”

Section: Joint Sequence-structure Decodermentioning

confidence: 99%

“…Nevertheless, these approaches are restricted to proteins with specific domain topologies, i.e., antibodies, and often suffer from high inference costs due to autoregressive sampling or annealed diffusion sampling (Song & Ermon, 2019;Luo et al, 2022). Very recently, Anand & Achim (2022) propose another diffusionbased generative model (Ho et al, 2020) for general protein sequence-structure co-design, where they adopt three diffusion models to generate structures, sequences, and rotamers of proteins in sequential order. Although applicable to proteins of all topologies, such a sequential generation strategy fails to cross-condition on sequence and structure, which might lead to inconsistent and fanciful proteins.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Protein Sequence and Structure Co-Design with Equivariant Translation

Shi¹,

Wang²,

Lu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Proteins are macromolecules that perform essential functions in all living organisms. Designing novel proteins with specific structures and desired functions has been a long-standing challenge in the field of bioengineering. Existing approaches generate both protein sequence and structure using either autoregressive models or diffusion models, both of which suffer from high inference costs. In this paper, we propose a new approach capable of protein sequence and structure co-design, which iteratively translates both protein sequence and structure into the desired state from random initialization, based on context features given a priori. Our model consists of a trigonometry-aware encoder that reasons geometrical constraints and interactions from context features, and a roto-translation equivariant decoder that translates protein sequence and structure interdependently. Notably, all protein amino acids are updated in one shot in each translation step, which significantly accelerates the inference process. Experimental results across multiple tasks show that our model outperforms previous state-of-the-art baselines by a large margin, and is able to design proteins of high fidelity as regards both sequence and structure, with running time orders of magnitude less than sampling-based methods.Preprint.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Preliminariesmentioning

confidence: 99%

Section: Joint Sequence-structure Decodermentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Protein Sequence and Structure Co-Design with Equivariant Translation

Shi¹,

Wang²,

Lu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The method was extensively evaluated experimentally [92]. Lastly, Protein Diffusion [94] leveraged diffusion models [95]- [97] popularized by generative image approaches [98] to train a model capable of generating sequence and structure based on a set of secondary structure input constraints.…”

Section: The Deep Learning Era Of Protein Sequence and Structure Gene...mentioning

confidence: 99%

From sequence to function through structure: deep learning for protein design

Ferruz

Heinzinger

Akdel³

et al. 2022

Preprint

View full text Add to dashboard Cite

The process of designing biomolecules, in particular proteins, is witnessing a rapid change in available tooling and approaches, moving from design through physicochemical force fields, to producing plausible, complex sequences fast via end-to-end differentiable statistical models. To achieve conditional and controllable protein design, researchers at the interface of artificial intelligence and biology leverage advances in natural language processing (NLP) and computer vision techniques, coupled with advances in computing hardware to learn patterns from growing biological databases, curated annotations thereof, or both. Once learned, these patterns can be leveraged to provide novel insights into mechanistic biology and the design of biomolecules. However, navigating and understanding the practical applications for the many recent protein design tools is complex. To facilitate this, we 1) document recent advances in deep learning (DL) assisted protein design from the last three years, 2) present a practical pipeline that allows to go from de novo-generated sequences to their predicted properties and web-powered visualization within minutes, and 3) leverage it to suggest a generated protein sequence which might be used to engineer a biosynthetic gene cluster to produce a molecular glue-like compound. Lastly, we discuss challenges and highlight opportunities for the protein design field.

show abstract

Fine-Tuned Transformer Model for Sentiment Analysis

Liu

Shuai

Zhang

et al. 2020

Knowledge Science, Engineering and Management

View full text Add to dashboard Cite

Foundational generative models should be traceable to protect their owners and facilitate safety regulation. To achieve this, traditional approaches embed identifiers based on supervisory trigger-response signals, which are commonly known as backdoor watermarks. They are prone to failure when the model is fine-tuned with nontrigger data. Our experiments show that this vulnerability is due to energetic changes in only a few 'busy' layers during fine-tuning. This yields a novel arbitrary-in-arbitrary-out (AIAO) strategy that makes watermarks resilient to fine-tuning-based removal. The trigger-response pairs of AIAO samples across various neural network depths can be used to construct watermarked subpaths, employing Monte Carlo sampling to achieve stable verification results. In addition, unlike the existing methods of designing a backdoor for the input/output space of diffusion models, in our method, we propose to embed the backdoor into the feature space of sampled subpaths, where a mask-controlled trigger function is proposed to preserve the generation performance and ensure the invisibility of the embedded backdoor. Our empirical studies on the MS-COCO, AFHQ, LSUN, CUB-200, and DreamBooth datasets confirm the robustness of AIAO; while the verification rates of other trigger-based methods fall from ∼90% to ∼70% after fine-tuning, those of our method remain consistently above 90%.

show abstract

Protein Structure and Sequence Generation with Equivariant Denoising Diffusion Probabilistic Models

Cited by 59 publications

References 21 publications

Protein Sequence and Structure Co-Design with Equivariant Translation

Protein Sequence and Structure Co-Design with Equivariant Translation

From sequence to function through structure: deep learning for protein design

Fine-Tuned Transformer Model for Sentiment Analysis

Contact Info

Product

Resources

About