OpenFold: Retraining AlphaFold2 yields new insights into its learning mechanisms and capacity for generalization

Abstract

AlphaFold2 revolutionized structural biology with the ability to predict protein structures with exceptionally high accuracy. Its implementation, however, lacks the code and data required to train new models. These are necessary to (i) tackle new tasks, like protein-ligand complex structure prediction, (ii) investigate the process by which the model learns, which remains poorly understood, and (iii) assess the model's generalization capacity to unseen regions of fold space. Here we report OpenFold, a fast, memory-efficient, and trainable implementation of AlphaFold2, and OpenProteinSet, the largest public database of protein multiple sequence alignments. We use OpenProteinSet to train OpenFold from scratch, fully matching the accuracy of AlphaFold2. Having established parity, we assess OpenFold's capacity to generalize across fold space by retraining it using carefully designed datasets. We find that OpenFold is remarkably robust at generalizing despite extreme reductions in training set size and diversity, including near-complete elisions of classes of secondary structure elements. By analyzing intermediate structures produced by OpenFold during training, we also gain surprising insights into the manner in which the model learns to fold proteins, discovering that spatial dimensions are learned sequentially. Taken together, our studies demonstrate the power and utility of OpenFold, which we believe will prove to be a crucial new resource for the protein modeling community.

TMvisDB: resource for transmembrane protein annotation and 3D visualization

“…The database and web application TMvisDB offers a straightforward search functionality and visualization interface. While several accurate structure prediction methods have been made available over the last year [14][15][16][17], we chose to enhance TMvisDB sequence annotations with AlphaFold2 [12] predictions that have been shown to perform well in structural analysis of transmembrane proteins (TMPs) [52], and have, therefore, been successfully applied as input by resources such as the TmAlphaFold database that collects alpha-helical TMPs [34].…”

tFold-Ab: Fast and Accurate Antibody Structure Prediction without Sequence Homologs

“…Sequence-based Protein Structure Prediction Since AlphaFold2 achieve remarkable performance at CASP14, some group are looking to speed up AlphaFold2 pipeline to reduce the computational expense, including model inference stage [2,7] and homologs search stage [25,13]. In nature a protein folds without knowledge of its sequence homologs, predicting protein structure based upon some non-natural conditions such as MSA or template does not reflect very well how a protein actually folds.…”

Efficient AlphaFold2 Training using Parallel Evoformer and Branch Parallelism

“…We make a comparison among AlphaFold2 (Jumper et al 2021), Open-Fold (Ahdritz et al 2021), HelixFold (Wang et al 2022) and UniFold (Li et al 2022), in terms of hardware, time cost of each step, training throughput and total training time as…”