In-Silico Staining from Bright-Field and Fluorescent Images Using Deep Learning

Waibel, Dominik J. E.; Tiemann, Ulf; Lupperger, Valerio; Semb, Henrik; Marr, Carsten

doi:10.1007/978-3-030-30508-6_15

Cited by 2 publications

(3 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this ‘ in silico labeling’ or ‘label-free microscopy’ approach, separate deep neural networks are trained to predict the likely distribution of specific organelle markers from transmitted light or other label-free images. Christiansen et al (2018) first proposed this method to label the nucleus and neurons; Ounkomol et al (2018) dramatically extended this approach by in silico labeling of multiple subcellular structures; further enhancements have been described ( Cooke et al , 2021 ; Waibel et al , 2019 ). These approaches used variations on the powerful U-Net model ( Falk et al , 2019 ; Ronneberger et al , 2015 ).…”

Section: Introductionmentioning

confidence: 99%

Improving and evaluating deep learning models of cellular organization

Sun

Abraham

et al. 2022

Bioinformatics

View full text Add to dashboard Cite

Motivation Cells contain dozens of major organelles and thousands of other structures, many of which vary extensively in their number, size, shape and spatial distribution. This complexity and variation dramatically complicates the use of both traditional and deep learning methods to build accurate models of cell organization. Most cellular organelles are distinct objects with defined boundaries that do not overlap, while the pixel resolution of most imaging methods is not sufficient to resolve these boundaries. Thus while cell organization is conceptually object-based, most current methods are pixel-based. Using extensive image collections in which particular organelles were fluorescently-labeled, deep learning methods can be used to build conditional autoencoder models for particular organelles. A major advance occurred with the use of a U-net approach to make multiple models all conditional upon a common reference, unlabeled image, allowing the relationships between different organelles to be at least partially inferred. Results We have developed improved GAN-based approaches for learning these models and have also developed novel criteria for evaluating how well synthetic cell images reflect the properties of real images. The first set of criteria measure how well models preserve the expected property that organelles do not overlap. We also developed a modified loss function that allows retraining of the models to minimize that overlap. The second set of criteria uses object-based modeling to compare object shape and spatial distribution between synthetic and real images. Our work provides the first demonstration that, at least for some organelles, deep learning models can capture object-level properties of cell images. Availability http://murphylab.cbd.cmu.edu/Software/2022_insilico. Supplementary information Supplementary data are available at Bioinformatics online.

show abstract

Section: Introductionmentioning

confidence: 99%

Improving and evaluating deep learning models of cellular organization

Sun

Abraham

et al. 2022

Bioinformatics

View full text Add to dashboard Cite

show abstract

“…In this "in silico labeling" or "label-free microscopy" approach, separate deep neural networks are trained to predict the likely distribution of specific organelle markers from transmitted light or other label-free images. Christiansen et al [2018] first proposed this method to label the nucleus and neurons; Ounkomol et al [2018], Cooke et al [2021], Waibel et al [2019] dramatically extended this approach by in silico labeling of multiple subcellular structures. These approaches used variations on the powerful U-Net model with a series of pooling and up-sampling operations that helps the model to capture image information at different scales [Ronneberger et al, 2015, Falk et al, 2019.…”

Section: Introductionmentioning

confidence: 99%

“…[2018], Cooke et al . [2021], Waibel et al . [2019] dramatically extended this approach by in silico labeling of multiple subcellular structures.…”

Section: Introductionmentioning

confidence: 99%

Improving and evaluating deep learning models of cellular organization

Sun

Abraham

et al. 2022

Preprint

View full text Add to dashboard Cite

Motivation: Cells contain dozens of major organelles and thousands of other structures, many of which vary extensively in their number, size, shape and spatial distribution. This complexity and variation dramatically complicates the use of both traditional and deep learning methods to build accurate models of cell organization. Most cellular organelles are distinct objects with defined boundaries that do not overlap, while the pixel resolution of most imaging methods is not sufficient to resolve these boundaries. Thus while cell organization is conceptually object-based, most current methods are pixel-based. Using extensive image collections in which particular organelles were fluorescently-labeled, deep learning methods can be used to build conditional autoencoder models for particular organelles. A major advance occurred with the use of a U-net approach to make multiple models all conditional upon a common reference, unlabeled image, allowing the relationships between different organelles to be at least partially inferred. Results: We have developed improved GAN-based approaches for learning these models and have also developed novel criteria for evaluating how well synthetic cell images reflect the properties of real images. The first set of criteria measure how well models preserve the expected property that organelles do not overlap. We also developed a modified loss function that allows retraining of the models to minimize that overlap. The second set of criteria uses object-based modeling to compare object shape and spatial distribution between synthetic and real images. Our work provides the first demonstration that, at least for some organelles, deep learning models can capture object-level properties of cell images.Availability: A Reproducible Research Archive containing all source code, generated images and analysis results will be made available at http://murphylab.cbd.cmu.edu/Software upon publication.

show abstract

In-Silico Staining from Bright-Field and Fluorescent Images Using Deep Learning

Cited by 2 publications

References 4 publications

Improving and evaluating deep learning models of cellular organization

Improving and evaluating deep learning models of cellular organization

Improving and evaluating deep learning models of cellular organization

Contact Info

Product

Resources

About