Estimation of Orientation and Camera Parameters from Cryo-Electron Microscopy Images with Variational Autoencoders and Generative Adversarial Networks

Miolane, Nina; Poitevin, Frédéric; Li, Yee-Ting; Holmes, Susan

doi:10.1109/cvprw50498.2020.00493

Cited by 15 publications

(16 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When applied on a ribosome dataset that had been previously carefully analyzed using a divide-and-conquer 3D classification approach, cryoDRGN showed the ability to directly map the relevant clusters on a low dimensional manifold that could then be further analyzed to understand how the different classes are topologically related. Despite their popularity, the use of neural networks for 3D reconstruction in cryo-EM is fairly new (see also [117,118]), suggesting promising directions for future research involving new learning architectures.…”

Section: Investigating Conformational Heterogeneitymentioning

confidence: 99%

Structural Heterogeneities of the Ribosome: New Frontiers and Opportunities for Cryo-EM

Poitevin

Kushner

et al. 2020

Molecules

View full text Add to dashboard Cite

The extent of ribosomal heterogeneity has caught increasing interest over the past few years, as recent studies have highlighted the presence of structural variations of the ribosome. More precisely, the heterogeneity of the ribosome covers multiple scales, including the dynamical aspects of ribosomal motion at the single particle level, specialization at the cellular and subcellular scale, or evolutionary differences across species. Upon solving the ribosome atomic structure at medium to high resolution, cryogenic electron microscopy (cryo-EM) has enabled investigating all these forms of heterogeneity. In this review, we present some recent advances in quantifying ribosome heterogeneity, with a focus on the conformational and evolutionary variations of the ribosome and their functional implications. These efforts highlight the need for new computational methods and comparative tools, to comprehensively model the continuous conformational transition pathways of the ribosome, as well as its evolution. While developing these methods presents some important challenges, it also provides an opportunity to extend our interpretation and usage of cryo-EM data, which would more generally benefit the study of molecular dynamics and evolution of proteins and other complexes.

show abstract

Section: Investigating Conformational Heterogeneitymentioning

confidence: 99%

Structural Heterogeneities of the Ribosome: New Frontiers and Opportunities for Cryo-EM

Poitevin

Kushner

et al. 2020

Molecules

View full text Add to dashboard Cite

show abstract

“…We are the first to propose an image translation based simulation method for cryo-ET 3D images. Although image translation has been used to simulate cryo-EM 2D images (Gupta et al, 2020b , 2021 ; Miolane et al, 2020 ), they are not directly comparable to our method as 3D cryo-ET and 2D cryo-EM images capture different kinds of information. One prior work applying GANs in a related space is Gupta et al ( 2020a ), in which a GAN is trained to perform single-particle cryogenic electron microscopy (Cryo-EM) reconstruction given a large number of Cryo-EM images.…”

Section: Introductionmentioning

confidence: 94%

CryoETGAN: Cryo-Electron Tomography Image Synthesis via Unpaired Image Translation

Zeng

et al. 2022

Front. Physiol.

View full text Add to dashboard Cite

Cryo-electron tomography (Cryo-ET) has been regarded as a revolution in structural biology and can reveal molecular sociology. Its unprecedented quality enables it to visualize cellular organelles and macromolecular complexes at nanometer resolution with native conformations. Motivated by developments in nanotechnology and machine learning, establishing machine learning approaches such as classification, detection and averaging for Cryo-ET image analysis has inspired broad interest. Yet, deep learning-based methods for biomedical imaging typically require large labeled datasets for good results, which can be a great challenge due to the expense of obtaining and labeling training data. To deal with this problem, we propose a generative model to simulate Cryo-ET images efficiently and reliably: CryoETGAN. This cycle-consistent and Wasserstein generative adversarial network (GAN) is able to generate images with an appearance similar to the original experimental data. Quantitative and visual grading results on generated images are provided to show that the results of our proposed method achieve better performance compared to the previous state-of-the-art simulation methods. Moreover, CryoETGAN is stable to train and capable of generating plausibly diverse image samples.

show abstract

“…The work in [45] uses neural networks to model the continuous generative factors of structural heterogeneity. Another work [46] uses a variational autoencoder trained using a discriminator-based objective to find a low-dimensional latent representation of the particles. These representations are then used to estimate the poses.…”

Section: Related Workmentioning

confidence: 99%

“…One approach would be to include a step that automatically spots and discards corrupted data so that the discriminator never gets to see them. Recent deep-learning-based approaches able to track outliers in data could prove useful in this regard [46]. main concept will be used in the second part.…”

Section: B Future Improvementsmentioning

confidence: 99%

CryoGAN: A New Reconstruction Paradigm for Single-Particle Cryo-EM Via Deep Adversarial Learning

Gupta

McCann

Donati

et al. 2021

IEEE Trans. Comput. Imaging

View full text Add to dashboard Cite

We present CryoGAN, a new paradigm for singleparticle cryo-electron microscopy (cryo-EM) reconstruction based on unsupervised deep adversarial learning. In singleparticle cryo-EM, the structure of a biomolecule needs to be reconstructed from a large set of noisy tomographic projections with unknown orientations. Current reconstruction techniques are based on a marginalized maximum-likelihood formulation that requires calculations over the set of all possible poses for each projection image, a computationally demanding procedure. Our approach is to seek a 3D structure that has simulated projections that match the real data in a distributional sense, thereby sidestepping pose estimation or marginalization. We prove that, in an idealized mathematical model of cryo-EM, this approach results in recovery of the correct structure. Motivated by distribution matching, we propose CryoGAN, a specialized GAN that consists of a 3D structure, a cryo-EM physics simulator, and a discriminator neural network. During reconstruction, the 3D structure is optimized so that its projections obtained through the simulator resemble real data (to the discriminator). Simultaneously, the discriminator is trained to distinguish real projections from simulated projections. CryoGAN takes as input only real projection images and the distribution of the cryo-EM imaging parameters. It involves neither prior training nor an initial estimation of the 3D structure. CryoGAN currently achieves a 10.8Å resolution on a realistic synthetic dataset. Preliminary results on experimental -galactosidase and 80S ribosome data demonstrate the ability of CryoGAN to exploit data statistics under standard experimental imaging conditions. We believe that this paradigm opens the door to a family of novel likelihood-free algorithms for cryo-EM reconstruction.

show abstract

Estimation of Orientation and Camera Parameters from Cryo-Electron Microscopy Images with Variational Autoencoders and Generative Adversarial Networks

Cited by 15 publications

References 21 publications

Structural Heterogeneities of the Ribosome: New Frontiers and Opportunities for Cryo-EM

Structural Heterogeneities of the Ribosome: New Frontiers and Opportunities for Cryo-EM

CryoETGAN: Cryo-Electron Tomography Image Synthesis via Unpaired Image Translation

CryoGAN: A New Reconstruction Paradigm for Single-Particle Cryo-EM Via Deep Adversarial Learning

Contact Info

Product

Resources

About