A Diffusion Model Predicts 3D Shapes from 2D Microscopy Images

Waibel, Dominik J. E.; Röell, Ernst; Rieck, Bastian; Giryes, Raja; Marr, Carsten

doi:10.48550/arxiv.2208.14125

Cited by 5 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The capacity of diffusion models to accurately create realistic visual data is profoundly impacting many computer vision applications 15 , including microscopic imaging, where overcoming the existing challenges to gather high-quality large training datasets is invaluable. Indeed, several studies already incorporate diffusion models to microscopy to reconstruct 3D biomolecule structures in Cryo-EM images 16 , predict 3D cellular structures out of 2D images 17 , or drug molecule design 18 , among others.…”

Section: Introductionmentioning

confidence: 99%

This microtubule does not exist: Super-resolution microscopy image generation by a diffusion model

Saguy,

Nahimov,

Lehrman

et al. 2023

Preprint

View full text Add to dashboard Cite

Generative models, such as diffusion models, have made significant advancements in recent years, enabling the synthesis of high-quality realistic data across various domains. Here, we explore the adaptation of a diffusion model to super-resolution microscopy images and train it on images from a publicly available database. We show that the generated images resemble experimental images, and that the generated images do not copy existing images from the training set. Additionally, we compare the performance of a deep-learning-based deconvolution method when trained on our generated data versus training on mathematical model based data and show superior reconstruction quality in means of spatial resolution. These findings demonstrate the potential contribution of generative diffusion models for microscopy tasks and pave the way for their future application in this field.

show abstract

Section: Introductionmentioning

confidence: 99%

This microtubule does not exist: Super-resolution microscopy image generation by a diffusion model

Saguy,

Nahimov,

Lehrman

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Then, the model learns to reverse this process by generating novel data starting from the noisy inputs (24). The usage of diffusion models in medical image generation includes (but is not limited to) brain magnetic resonance images (25) (26), microscopic blood cells images (27), positron emission tomography heart images (28), and chest x-ray (29). A comprehensive review on the subject has been recently published (30).…”

Section: Introductionmentioning

confidence: 99%

Generating synthetic data in digital pathology through diffusion models: a multifaceted approach to evaluation

Pozzi,

Noei,

Robbi

et al. 2023

Preprint

View full text Add to dashboard Cite

Synthetic data has recently risen as a new precious item in the computational pathologist’s toolbox, supporting several tasks such as helping with data scarcity or augmenting training set in deep learning. Nonetheless, the use of such novel resources requires a carefully planned construction and evaluation, to avoid pitfalls such as the generation of clinically meaningless artifacts.As the major outcome described in the current manuscript, a novel full stack pipeline is introduced for the generation and evaluation of synthetic pathology data powered by a diffusion model. The workflow features, as characterizing elements, a new multifaceted evaluation strategy with an embedded explainability procedure effectively tackling two critical aspects of the use of synthetic data in health-related domains.An ensemble-like strategy is adopted for the evaluation of the produced data, with the threefold aim of assessing the similarity of real and synthetic data through a set of well-established metrics, evaluating the practical usability of the generated images in deep learning models complemented by explainable AI methods, and validating their histopathological realism through a dedicated questionnaire answered by three professional pathologists.The pipeline is demonstrated on the public GTEx dataset of 650 WSIs, including five different tissues, conditioning the training step of the underlying diffusion model. An equal number of tiles from each of these five tissues are then generated. Finally, the reliability of the generated data is assessed using the proposed evaluation pipeline, with encouraging results. We show that each of these evaluation steps are necessary as they provide complementary information on the generated data’s quality.Overall, all the aforementioned features characterize the proposed workflow as a fully-fledged solution for generative AI in digital pathology representing a potentially useful tool for the digital pathology community in their transition towards digitalization and data-driven modeling.

show abstract

“…Generative visual deep learning has greatly influenced the biomedical AI landscape following the impressive performance of generative adversarial networks (GANs) [20,21] in image reconstruction and processing. The advent of diffusion [22] expanded the applicability of generative AI, now encompassing areas such as medical imaging [23] and computational microscopy [24,25]. Normalising flows, a class of generative models less common in biomedical research, are popular in the physical sciences [26,27] for their strong statistical basis.…”

Section: Introductionmentioning

confidence: 99%

Multi-scale tissue fluorescence mapping with fibre optic ultraviolet excitation and generative modelling

Yong

Tan

et al. 2022

Preprint

View full text Add to dashboard Cite

Rapidly resolving microstructure and function from thick biological samples is a critical requirement from basic research to clinical diagnosis. The need for physical sectioning leads to limitations in the speed, cost, or ease of sample preparation and imaging. We introduce a fluorescence microscope with deep ultraviolet illumination via omnidirectional fibre optics, based on the concept of microscopy with ultraviolet surface excitation (MUSE). We further exploited precise control of directional illumination to produce 3D reconstructions of surface microtopography from 2D images. Our technology is applicable to diverse sample types, ranging from single cells and 3D cultures to large tissue samples while utilising conventional, widely available objective lenses in a compact and portable implementation. We demonstrated the use of our microscope with important fluorescent assays through quantitative viability studies in single cells to whole organs, organ-level studies comparable to standard H&E histology, and multiplexed immunohistochemistry staining. We also explored the potential for microtopography to provide additional insight into vascular and tissue texture and heterogeneity studies. This technology fills long-standing technological gaps in both the laboratory and clinic for a general-purpose microscope that is simple, portable, and cheap that can provide near real-time biological insights.

show abstract

A Diffusion Model Predicts 3D Shapes from 2D Microscopy Images

Cited by 5 publications

References 0 publications

This microtubule does not exist: Super-resolution microscopy image generation by a diffusion model

This microtubule does not exist: Super-resolution microscopy image generation by a diffusion model

Generating synthetic data in digital pathology through diffusion models: a multifaceted approach to evaluation

Multi-scale tissue fluorescence mapping with fibre optic ultraviolet excitation and generative modelling

Contact Info

Product

Resources

About