A Self-Supervised Machine Learning Approach for Objective Live Cell Segmentation and Analysis

Abstract

Machine learning algorithms hold the promise of greatly improving live cell image analysis by way of (1) analyzing far more imagery than can be achieved by more traditional manual approaches and (2) by eliminating the subjective nature of researchers and diagnosticians selecting the cells or cell features to be included in the analyzed data set. Currently, however, even the most sophisticated model based or machine learning algorithms require user supervision, meaning the subjectivity problem is not removed but rather incorporated into the algorithm’s initial training steps and then repeatedly applied to the imagery. To address this roadblock, we have developed a self-supervised machine learning algorithm that recursively trains itself directly from the live cell imagery data, thus providing objective segmentation and quantification. The approach incorporates an optical flow algorithm component to self-label cell and background pixels for training, followed by the extraction of additional feature vectors for the automated generation of a cell/background classification model. Because it is self-trained, the software has no user-adjustable parameters and does not require curated training imagery. The algorithm was applied to automatically segment cells from their background for a variety of cell types and five commonly used imaging modalities - fluorescence, phase contrast, differential interference contrast (DIC), transmitted light and interference reflection microscopy (IRM). The approach is broadly applicable in that it enables completely automated cell segmentation for long-term live cell phenotyping applications, regardless of the input imagery’s optical modality, magnification or cell type.

Interfacing Live Cells with Surfaces: A Concurrent Control Technique for Quantifying Surface Ligand Activity

“…serum, BSA) which can alter cellular phenotype in unexpected ways 30 . While phase contrast microscopy was employed in this work, the surfaces are compatible with other commonly used optical modalities such as fluorescence, DIC, interference reflection microscopy, and transmitted light illumination 34 .…”

“…For each image, the algorithm outputs metadata which includes the outlines of the segmented cells as well as morphological parameters such as cell area and circularity. 34…”

Single-shot self-supervised object detection in microscopy

“…LodeSTAR builds on geometric deep learning 13 and the recent surge of self-supervised object tracking methods [14][15][16][17][18][19][20][21] to create a self-supervised (or more precisely, self-distillative) object-detection neural network optimized for microscopy data. Specifically, we exploit the fact that a neural network that is equivariant to rotations and translations (i.e., a neural network for which a roto-translational transformation of the input image produces an equivalent roto-translation of the prediction) operates as an object detector (see Methods, "Theory of geometric self-distillation").…”

Automated detection of patterned single-cells within hydrogel using deep learning

“…Single cells need to be automatically characterized to reduce analysis time and efficiently reveal new findings. With the advancement in DL (deep learning) and artificial intelligence, there has been a development in intelligent tools that can perform image analysis on microscopic data 20 , 21 and eventually automate the entire process of data collection and artifact detections 22 , 23 . Recent advancements in the application of deep learning networks range from graph-based cell detection and score estimation, cell counting 20 , and cell segmentation 21 , to fully developed pipelines that characterize the development of cells under given conditions 22 .…”

“…With the advancement in DL (deep learning) and artificial intelligence, there has been a development in intelligent tools that can perform image analysis on microscopic data 20 , 21 and eventually automate the entire process of data collection and artifact detections 22 , 23 . Recent advancements in the application of deep learning networks range from graph-based cell detection and score estimation, cell counting 20 , and cell segmentation 21 , to fully developed pipelines that characterize the development of cells under given conditions 22 . The functionalities of DL are not restricted to the physical morphology, rather literature shows that image-based cell phenotyping has also been realized in recent years 24 .…”