Deep learning for cellular image analysis

Moen, Erick; Bannon, Dylan; Kudo, Tetsuichi; Graf, William D.; Covert, Markus W.; Valen, David Van

doi:10.1038/s41592-019-0403-1

Cited by 909 publications

(699 citation statements)

References 137 publications

Supporting

Mentioning

659

Contrasting

Unclassified

Order By: Relevance

“…One major concern with deep learning-based image processing is accuracy, and in particular the possibility of false positives (aka "hallucinations") 3,4,19,20 . As mentioned above, 2 nm pixel SBFSEM datasets are beyond the capabilities for our samples and detector, precluding the generation of ground truth validation images for our SBFSEM data.…”

Section: Resultsmentioning

confidence: 99%

“…It is important to consider that any output from a deep learning super-resolution model is a prediction, is never 100% accurate, and is always highly dependent on proper correspondence between the training versus testing data 3,4,20,22 . Whether the level of accuracy of a given model for a given dataset is sufficient is ultimately dependent on whether the tolerance for error in the measurement being made is higher than the actual error.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning-Based Point-Scanning Super-Resolution Imaging

Fang

Monroe²,

Novak

et al. 2019

Preprint

View full text Add to dashboard Cite

Point scanning imaging systems (e.g. scanning electron or laser scanning confocal microscopes) are perhaps the most widely used tools for high resolution cellular and tissue imaging. Like all other imaging modalities, the resolution, speed, sample preservation, and signal-to-noise ratio (SNR) of point scanning systems are difficult to optimize simultaneously. In particular, point scanning systems are uniquely constrained by an inverse relationship between imaging speed and pixel resolution. Here we show these limitations can be mitigated via the use of deep learning-based super-sampling of undersampled images acquired on a point-scanning system, which we termed point-scanning super-resolution (PSSR) imaging. Oversampled, high SNR ground truth images acquired on scanning electron or Airyscan laser scanning confocal microscopes were 'crappified' to generate semi-synthetic training data for PSSR models that were then used to restore real-world undersampled images. Remarkably, our EM PSSR model could restore undersampled images acquired with different optics, detectors, samples, or sample preparation methods in other labs. PSSR enabled previously unattainable 2 nm resolution images with our serial block face scanning electron microscope system. For fluorescence, we show that undersampled confocal images combined with a multiframe PSSR model trained on Airyscan timelapses facilitates Airyscan-equivalent spatial resolution and SNR with ~100x lower laser dose and 16x higher frame rates than corresponding high-resolution acquisitions. In conclusion, PSSR facilitates point-scanning image acquisition with otherwise unattainable resolution, speed, and sensitivity. Fig. 1 | Restoration of semi-synthetic and real-world EM testing data using PSSR model trained on semi-synthetically generated training pairs. a, Overview of the general workflow.Training pairs were semi-synthetically created by applying a degrading function to the HR images taken from a scanning electron microscope in transmission mode (tSEM) to generate LR counterparts (left column). Semi-synthetic pairs were used as training data through a dynamic ResNet-based U-Net architecture (middle column). Real-world LR and HR image pairs were both manually acquired under a SEM (right column). The output from PSSR (LR-PSSR) when LR is served as input is then compared to HR to evaluate the performance of our trained model. b, Restoration performance on semi-synthetic testing pairs from tSEM. Shown is the same field of view of a representative bouton region from the synthetically created LR input with the pixel size of 8 nm (left column), a 16x bilinear upsampled image with 2 nm pixel size (second column), 16x PSSR upsampled result with 2 nm pixel size (third column) and the HR ground truth acquired at the microscope with the pixel size of 2 nm (fourth column). A close view of the same vesicle in each image is highlighted. The Peak-Signal-to-Noise-Ratio (PSNR) and the Structural Similarity (SSIM) quantification of the semi-synthetic testing sets are shown (right). c, Restor...

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Deep Learning-Based Point-Scanning Super-Resolution Imaging

Fang

Monroe²,

Novak

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…But when it is really used in life and production, the sampled pictures have different 73 characteristics with the sampling conditions and disease types. A series of algorithms including 74 pre-processing, segmentation and feature extraction are designed for each specific project (Boucher et 75 al., 1998;Moen et al, 2019;Van Valen et al, 2016). The level of the designers and the selection of 76 parameters in the algorithm can easily affect cell recognition.…”

Section: Introductionmentioning

confidence: 99%

The Study on Computer Vision-Assisted Cell Bank Construction and Screening & Classification

Yang

Chen

Bai

et al. 2019

Preprint

View full text Add to dashboard Cite

8Computer vision technology plays an important role in screening and culturing cells. This paper 9proposes a method to construct a helper cell library based on cell image segmentation and screening. 10 Firstly, cell culture and image acquisition were carried out. The main content is to use laboratory 11 conditions to carry out different cell types. Through careful observation of the whole process of cell 12 proliferation and passage, the representative pictures of different stages were taken. Analysis and 13 summary of the relevant morphology, texture, color characteristics. Secondly, computer vision 14 technology is used to segment cells and extract the main features such as cell perimeter and area. 15 Explore the automatic information extraction method of cell bank, and complete the image 16 segmentation of individual cell image from the whole picture. Finally, the cells were screened and 17 identified. Investigate different pattern recognition methods and neural network structures, and 18 prepare pictures of various cell pictures. The corresponding neural network and prediction program 19 are constructed. This paper proposes an automatic image processing method for each image category 20 in cell culture cycle, which improves the automation of production process. At the same time, 21 compared with the design of a single algorithm for a certain type of cell, different algorithm design 22 ideas are proposed for three types of pictures with different characteristics, which is closer to the 23 dynamic change of cell morphology in the process of cell culture. This research has important 24 application prospects in promoting cell factory research, cell bank construction and automatic 25 screening. 26

show abstract

“…Chief among these new tools is deep learning, a set of machine learning tools that can learn effective representation from data in a supervised or unsupervised manner. These methods are more accurate than prior approaches 3 and can automate the image classification and image segmentation tasks that have formed the bedrock of single-cell analysis 3 . Their ability to extract latent information from images has also enabled previously unforeseen analyses of cellular function and behavior.…”

Section: Introductionmentioning

confidence: 99%

DeepCell Kiosk: Scaling deep learning-enabled cellular image analysis with Kubernetes

Bannon

Moen

Schwartz

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

Deep learning is transforming the ability of life scientists to extract information from images. These techniques have better accuracy than conventional approaches and enable previously impossible analyses. As the capability of deep learning methods expands, they are increasingly being applied to large imaging datasets. The computational demands of deep learning present a significant barrier to large-scale image analysis. To meet this challenge, we have developed DeepCell 2.0, a platform for deploying deep learning models on large imaging datasets (>10 5megapixel images) in the cloud. This software enables the turnkey deployment of a Kubernetes cluster on all commonly used operating systems. By using a microservice architecture, our platform matches computational operations with their hardware requirements to reduce operating costs. Further, it scales computational resources to meet demand, drastically reducing the time necessary for analysis of large datasets. A thorough analysis of costs demonstrates that cloud computing is economically competitive for this application. By treating hardware infrastructure as software, this work foreshadows a new generation of software packages for biology in which computational resources are a dynamically allocated resource.

show abstract

Deep learning for cellular image analysis

Cited by 909 publications

References 137 publications

Deep Learning-Based Point-Scanning Super-Resolution Imaging

Deep Learning-Based Point-Scanning Super-Resolution Imaging

The Study on Computer Vision-Assisted Cell Bank Construction and Screening & Classification

DeepCell Kiosk: Scaling deep learning-enabled cellular image analysis with Kubernetes

Contact Info

Product

Resources

About