Interpretable deep learning of label-free live cell images uncovers functional hallmarks of highly-metastatic melanoma

Abstract

Deep convolutional neural networks have emerged as a powerful technique to identify hidden patterns in complex cell imaging data. However, these machine learning techniques are often criticized as uninterpretable "black-boxes" -lacking the ability to provide meaningful explanations for the cell properties that drive the machine's prediction. Here, we demonstrate that the latent features extracted from label-free live cell images by an adversarial auto-encoding deep convolutional neural network capture subtle details of cell appearance that allow classification of melanoma cell states, including the metastatic efficiency of seven patientderived xenograft models that reflect clinical outcome. Although trained exclusively on patientderived xenograft models, the same classifier also predicted the metastatic efficiency of immortalized melanoma cell lines suggesting that the latent features capture properties that are specifically associated with the metastatic potential of a melanoma cell regardless of its origin.We used the autoencoder to generate "in-silico" cell images that amplified the cellular features driving the classifier of metastatic efficiency. These images unveiled pseudopodial extensions and increased light scattering as functional hallmarks of metastatic cells. We validated this interpretation by analyzing experimental image time-lapse sequences in which melanoma cells spontaneously transitioned between states indicative of low and high metastatic efficiency.Together, this data is an example of how the application of Artificial Intelligence supports the identification of processes that are essential for the execution of complex integrated cell functions but are too subtle to be identified by a human expert. -Gurion University of the Negev, Israel (to AZ). We thank Sean Morrison for PDX-derived cell models. We thank Andrew R. Cohen for LEVER. Author ContributionAZ, ESW and GD conceived the study. ESW designed the experimental assay, optimized culture conditions for PDX cells, and trained AN and JC to perform the experiments. AN performed all live imaging and mouse experiments. JC performed additional experiments. UE generated the PDX samples. AZ and ARJ developed analytic tools, analyzed, and interpreted the data. AZ drafted the manuscript. All authors wrote and edited the manuscript and approved its content.GD mentored all authors.

Single-cell cytometry via multiplexed fluorescence prediction by label-free reflectance microscopy

“…Meanwhile, it has been shown that DL techniques can holistically capture complex structural features for classification. This has found broad applications in detecting cell types (12)(13)(14), cell states (15)(16)(17)(18)22), drug response (19), and stem cell lineage (20). By fully leveraging the label-free and high multiplexing nature of our technique, it can potentially generate significant impacts in imaging cytometry by offering unprecedented information content and discovering new compound morphological features necessitating multiplexed fluorescence readout.…”

“…By doing so, multiple subcellular structures and cell states can be revealed simultaneously without physical labeling. While previous work has shown that DL models can disentangle the complex structures captured in the label-free data and make in-silico fluorescence labeling with high accuracy (10)(11)(12) or holistically capture "hidden" structural features that are not easily perceived or described (13)(14)(15)(16)(17)(18)(19)(20)(21)(22), these results are fundamentally limited by the weak structural contrast from the transmission modes that contain only forward scattering information. By exploiting the enhanced resolution and sensitivity in the backscattering data, we demonstrate a dramatic increase in the fluorescence prediction accuracy with up to 3× improvement as compared to the current state-of-the-art.…”

DynaMorph: self-supervised learning of morphodynamic states of live cells

“…We used self-supervised deep learning ( Figure 1E) to build an unbiased model of microglia morphology. Recent work has demonstrated that self-supervised learning with autoencoders (5,6) can provide a quantitative model of cell morphology informed by all of data.…”

“…Recent work on analysis of morphological states of cells has relied on images of fixed cells labeled with a panel of fluorescent markers (1), live three-dimensional imaging of the membrane labeled with genetic markers (2), and phase contrast imaging of live cells (3)(4)(5)(6). The morphological states have been analyzed with low dimensional representations computed with geometric or biophysical models (3,(7)(8)(9)(10)(11), supervised learning of morphological labels (4,(12)(13)(14)(15)(16)(17), and, recently, self-supervised learning of latent representations of morphology (5,6). These analytical approaches have been inspired by the need for quantitative descriptions of specific, complex biological functions, such as motility of single cells (2,3,7,8,18), collective cell migration (9,11), cell cycle (4,12,13), spatial gene expression (17), and spatial protein expression (14,16).In addition, data-driven integration of the morphology and gene expression (13,17,(19)(20)(21)(22) is now enabling rapid analysis of functional roles of genes.…”

“…Microglia survey brain parenchyma with their highly motile processes and respond to changes in brain homeostasis by altering cell morphology and motility (26,27), but large-scale features of their dynamic behavior are not well characterized partly due to the lack of tractable molecular labeling tools. For other types of motile cells, either learned features or the manually curated features (3,5,6,18) have been used to describe complex cell dynamics, including random search, persistent migration towards target, changes in cell shape due to interaction with other cells and targets, and endocytosis, among others. However, changes in the expression of a small number of genes or morphological features of relevance to the other cell types may not reflect the full spectrum of microglia phenotypes and functions.…”