Deep neural network automated segmentation of cellular structures in volume electron microscopy

Abstract

Volume electron microscopy is an important imaging modality in contemporary cell biology. Identification of intracellular structures is a laborious process limiting the effective use of this potentially powerful tool. We resolved this bottleneck with automated segmentation of intracellular substructures in electron microscopy (ASEM), a new pipeline to train a convolutional neural network to detect structures of a wide range in size and complexity. We obtained dedicated models for each structure based on a small number of sparsely annotated ground truth images from only one or two cells. Model generalization was improved with a rapid, computationally effective strategy to refine a trained model by including a few additional annotations. We identified mitochondria, Golgi apparatus, endoplasmic reticulum, nuclear pore complexes, caveolae, clathrin-coated pits, and vesicles imaged by focused ion beam scanning electron microscopy. We uncovered a wide range of membrane–nuclear pore diameters within a single cell and derived morphological metrics from clathrin-coated pits and vesicles, consistent with the classical constant-growth assembly model.

Three-dimensional ultrastructure analysis of organelles in injured motor neuron

“…The main strategy for auto-segmentation is a three-dimensional convolutional neural network (CNN) architecture based on a three-dimensional U-Net (Çiçek et al 2016;Falk et al 2019). Some developing tools have been released by an open source, such as a repository providing large quantities of reliable data, codes, and trained models (Heinrich et al 2021), a new pipeline to train a CNN effectively (Gallusser et al 2023), improved DL platforms (Suga et al 2021), and so on. However, more challenges remain in an ongoing effort to reduce human labour for the generation of training data, proofreading predictions, and reducing computation costs.…”

CebraEM: A practical workflow to segment cellular organelles in volume SEM datasets using a transferable CNN-based membrane prediction

“…In most cases, the efforts focused on obtaining a high segmentation quality for just one organelle, like mitochondria (24)(25)(26)(27), or nuclear membrane (17). More recently, DL methods were successfully applied to full organelle segmentation (28,29). Using DL for segmenting organelles in EM comes with a set of additional challenges.…”

“…Another way to increase the efficiency of ground truth annotation is the usage of semi-automated approaches (13,31). To further facilitate the training procedure and simplify setting up deep learning models, many approaches introduce pretrained network architectures (28,29) or create readily implemented frameworks for training (32)(33)(34)(35). Despite those efforts, training deep learning models effectively still requires a certain level of knowledge in machine learning, which might discourage non-experts from training with their own data.…”

Out With a Bang: Celebrating Global Chemical Biology

“…Tomas Kirchhausen (Harvard Medical School, USA) started with a retrospective to the time 20 years ago before the NCCR started, when they discovered the compound Dynasore, a very potent Dynamin inhibitor . He then presented his inspiring latest research using large scale data visualization to achieve automated recognition of organelles from FIB-SEM data for both cultured cells and tissue using deep learning-aided image processing . The final part of his presentation was dedicated to his contribution to the research on SARS-CoV-2, where his laboratory characterized the infection route for the virus and helped discover a simple peptide exhibiting highly inhibitory properties against infections against SARS-CoV-2 variants to date …”

“…25 He then presented his inspiring latest research using large scale data visualization to achieve automated recognition of organelles from FIB-SEM data for both cultured cells and tissue using deep learning-aided image processing. 26 The final part of his presentation was dedicated to his contribution to the research on SARS-CoV-2, where his laboratory characterized the infection route for the virus 27 and helped discover a simple peptide exhibiting highly inhibitory properties against infections against SARS-CoV-2 variants to date. 28 After a restoring lunch break, the afternoon program was started by Prof. Paola Picotti (ETH Zurich, Switzerland), who shared her lab's latest results in revolutionizing proteomics to bring structural proteome snapshots into functional proteomics screens.…”