SimSearch: A Human-in-The-Loop Learning Framework for Fast Detection of Regions of Interest in Microscopy Images

Abstract

Large-scale microscopy-based experiments often result in images with rich but sparse information content. An experienced microscopist can visually identify regions of interest (ROIs), but this becomes a cumbersome task with large datasets. Here we present SimSearch, a framework for quick and easy user-guided training of a deep neural model aimed at fast detection of ROIs in large-scale microscopy experiments. Methods: The user manually selects a small number of patches representing different classes of ROIs. This is followed by feature extraction using a pre-trained deep-learning model, and interactive patch selection pruning, resulting in a smaller set of clean (user approved) and larger set of noisy (unapproved) training patches of ROIs and background. The pre-trained deep-learning model is thereafter first trained on the large set of noisy patches, followed by refined training using the clean patches. Results: The framework is evaluated on fluorescence microscopy images from a large-scale drug screening experiment, brightfield images of immunohistochemistry-stained patient tissue samples, and malaria-infected human blood smears, as well as transmission electron microscopy images of cell sections. Compared to state-of-the-art and manual/visual assessment, the results show similar performance with maximal flexibility and minimal a priori information and user interaction. Conclusions: SimSearch quickly adapts to different data sets, which demonstrates the potential to speed up many microscopy-based experiments based on a small amount Manuscript

Visualization and quality control tools for large-scale multiplex tissue analysis in TissUUmaps3

“…The goal of the analysis is to quantify this translocation of GFP, or more specifically, to measure the fraction of cells in an image that have nuclear or cytoplasmic GFP expression. In the evaluation of the ClassV&QC plugin, we compared results achieved by two different, fully automated classification methods; SimSearch (30) and CellProfiler (19,20) . CellProfiler classifies cells by first identifying individual cell nuclei, and then extracting measurements such as staining intensities from the surrounding area to assign cell classes.…”

Visualization & Quality Control Tools for Large-scale Multiplex Tissue Analysis in TissUUmaps 3

“…The goal of the analysis is to quantify this translocation of GFP, or more specifically, to measure the fraction of cells in an image that have nuclear or cytoplasmic GFP expression. In the evaluation of the ClassV&QC plugin, we compared results achieved by two different, fully automated classification methods; SimSearch [29] and CellProfiler [18, 19]. In this case, the cell IDs of the two approaches do not agree since CellProfiler’s cell identification is done independently from cell classification, and SimSearch’s cell identification is done simultaneously with cell classification.…”

“…The goal of the analysis is to quantify this translocation of GFP, or more specifically, to measure the fraction of cells in an image that have nuclear or cytoplasmic GFP expression. In the evaluation of the ClassV&QC plugin, we compared results achieved by two different, fully automated classification methods; SimSearch [30] and CellProfiler [19, 20]. CellProfiler classifies cells by first identifying individual cell nuclei, and then extracting measurements such as staining intensities from the surrounding area to assign cell classes.…”

Data Management of Scientific Applications in a Reinforcement Learning-Based Hierarchical Storage System

“…Here, SimSearch (Gupta, Sabirsh, Wählby, & Sintorn, 2022) is used to generate the counts of the cells with three categories: ''GFP in cytoplasm'', ''GFP in nuclei'', and ''No GFP expression''. SimSearch is a deep learning based ROI (Region Of Interest) detection framework for quickly annotating microscopy dataset.…”