From Shallow to Deep: Exploiting Feature-Based Classifiers for Domain Adaptation in Semantic Segmentation

Abstract

The remarkable performance of Convolutional Neural Networks on image segmentation tasks comes at the cost of a large amount of pixelwise annotated images that have to be segmented for training. In contrast, feature-based learning methods, such as the Random Forest, require little training data, but rarely reach the segmentation accuracy of CNNs. This work bridges the two approaches in a transfer learning setting. We show that a CNN can be trained to correct the errors of the Random Forest in the source domain and then be applied to correct such errors in the target domain without retraining, as the domain shift between the Random Forest predictions is much smaller than between the raw data. By leveraging a few brushstrokes as annotations in the target domain, the method can deliver segmentations that are sufficiently accurate to act as pseudo-labels for target-domain CNN training. We demonstrate the performance of the method on several datasets with the challenging tasks of mitochondria, membrane and nuclear segmentation. It yields excellent performance compared to microscopy domain adaptation baselines, especially when a significant domain shift is involved.

A deep learning-based strategy for producing dense 3D segmentations from sparsely annotated 2D images

“…Our method works reliably on small volumes for all initial amounts of sparse image annotations and all investigated datasets, suggesting general applicability. The use of pseudo, incomplete, or imprecise labels as training data is also not novel; neither is the idea of iterative bootstrapping to refine a model by acquiring more segmented data for training [23][24][25][26] . Others recently demonstrated an end-to-end pipeline that learned to correct swift and incomplete annotations like skeletons and seeds 27 .…”

BioImage Model Zoo: A Community-Driven Resource for Accessible Deep Learning in BioImage Analysis

“…In either case, only a minimal amount of simple Python code is needed to integrate a pipeline of multiple networks with a user-friendly GUI. 44 . is available in the Zoo.…”

“…5). Here, we show how the pre-trained models of the Zoo can be exploited for domain adaptation through the approach proposed by Matskevych et al 44 . We use their trained model for mitochondria segmentation in electron microscopy (EM) already available in the BioImage Model Zoo (model id: 10.5281/zenodo.6406756 or hiding-blowfish) and the images from the MitoEM challenge 45 .…”

“…Figure 5. Domain adaptation for EM mitochondria segmentation with pre-trained models from the Zoo; (a) We show several options for interactive training of a shallow classifier (i.e., Random Forest) that segments mitochondria in new EM data: Fiji plugins "Trainable Weka segmentation" and "Labkit'' and the pixel classification workflow in ilastik; (b) The domain adaptation model trained with the method from Matskevych et al44 . is available in the Zoo.…”