OpSeF: Open Source Python Framework for Collaborative Instance Segmentation of Bioimages

Abstract

Various pre-trained deep learning models for the segmentation of bioimages have been made available as developer-to-end-user solutions. They are optimized for ease of use and usually require neither knowledge of machine learning nor coding skills. However, individually testing these tools is tedious and success is uncertain. Here, we present the Open Segmentation Framework (OpSeF), a Python framework for deep learningbased instance segmentation. OpSeF aims at facilitating the collaboration of biomedical users with experienced image analysts. It builds on the analysts' knowledge in Python, machine learning, and workflow design to solve complex analysis tasks at any scale in a reproducible, well-documented way. OpSeF defines standard inputs and outputs, thereby facilitating modular workflow design and interoperability with other software. Users play an important role in problem definition, quality control, and manual refinement of results. OpSeF semi-automates preprocessing, convolutional neural network (CNN)based segmentation in 2D or 3D, and postprocessing. It facilitates benchmarking of multiple models in parallel. OpSeF streamlines the optimization of parameters for preand postprocessing such, that an available model may frequently be used without retraining. Even if sufficiently good results are not achievable with this approach, intermediate results can inform the analysts in the selection of the most promising CNNarchitecture in which the biomedical user might invest the effort of manually labeling training data. We provide Jupyter notebooks that document sample workflows based on various image collections. Analysts may find these notebooks useful to illustrate common segmentation challenges, as they prepare the advanced user for gradually taking over some of their tasks and completing their projects independently. The notebooks may also be used to explore the analysis options available within OpSeF in an interactive way and to document and share final workflows. Currently, three mechanistically distinct CNN-based segmentation methods, the U-Net implementation used in Cellprofiler 3.0, StarDist, and Cellpose have been integrated within OpSeF. The addition of new networks requires little; the addition of new models requires no coding skills. Thus, OpSeF might soon become both an interactive model repository, in which pre-trained models might be shared, evaluated, and reused with ease.

User-Accessible Machine Learning Approaches for Cell Segmentation and Analysis in Tissue

“…Furthermore, deep learning networks are available through online repositories such as github.com and modelzoo.co. An exciting development is the recent set of publications that have defined one-stop-shops for deep learning models and accessible tools for using and training existing deep learning networks ( Iudin et al, 2016 ; Berg et al, 2019 ; Rasse et al, 2020 ; Gómez-de-Mariscal et al, 2021 ; von Chamier et al, 2021 , p. 4). This includes the integration of segmentation tools with online repositories of trained deep learning networks that can be easily downloaded and tested on cells and modalities of interest.…”

Paving the Way: Contributions of Big Data to Apicomplexan and Kinetoplastid Research

“…We envisage that a step to achieve this is the development of user-friendly environments that allow users with little or no coding experience to use these resources. Recent successful examples include LOBSTER ( Tosi et al., 2020 ), BIAFLOWS ( Rubens et al., 2020 ), OpSeF ( Rasse et al., 2020 ), and ZeroCostDL4Mic ( von Chamier et al., 2021 ). Moreover, specialists are needed to bridge the two disciplines (artificial intelligence in imaging, and biology).…”

Cell Tracking in 3D using deep learning segmentations

“…Several methods have been proposed for such automated detection and segmentation tasks such as the traditional intensity based thresholding, watershed transform [BM18] and of recent machine learning methods based on random-forest classifiers and support vector machines [BKK + 19]. It was shown in [RHH20] that conventional computer vision and machine learning based techniques alone will almost always lead to sub-optimal segmentation and that methods based on deep learning have improved the accuracy of segmentation for natural and biomedical images alike. For the purpose of semantic segmentation U-Net [RFB15] has emerged as the most widely used network for biological applications.…”