Implicit Neural Representations for Generative Modeling of Living Cell Shapes

Abstract

Methods allowing the synthesis of realistic cell shapes could help generate training data sets to improve cell tracking and segmentation in biomedical images. Deep generative models for cell shape synthesis require a light-weight and flexible representation of the cell shape. However, commonly used voxel-based representations are unsuitable for high-resolution shape synthesis, and polygon meshes have limitations when modeling topology changes such as cell growth or mitosis. In this work, we propose to use level sets of signed distance functions (SDFs) to represent cell shapes. We optimize a neural network as an implicit neural representation of the SDF value at any point in a 3D+time domain. The model is conditioned on a latent code, thus allowing the synthesis of new and unseen shape sequences. We validate our approach quantitatively and qualitatively on C. elegans cells that grow and divide, and lung cancer cells with growing complex filopodial protrusions. Our results show that shape descriptors of synthetic cells resemble those of real cells, and that our model is able to generate topologically plausible sequences of complex cell shapes in 3D+time.

Denoising diffusion probabilistic models for generation of realistic fully-annotated microscopy image datasets

“…For all datasets, annotation masks are either obtained from manual annotations, automatically obtained unrefined segmentation or simulation approaches, to demonstrate use cases with various conditions. Multiple sophisticated approaches for automated simulation of cellular structures have already been proposed, ranging from physics-based methods [34], statistical shape-models [10,11] and spherical harmonics [10,35], to deep learning-based methods [12,36,37]. For simplicity, this work focuses on simulation approaches utilizing basic geometrical functions to create cellular structures.…”

“…Annotation efforts are reduced by automated data augmentation approaches [3][4][5] and tweaked segmentation pipelines [6,7], which help to ease the challenge, but still demand a small set of fully-annotated image data as a basis. Alternatively, automated simulation approaches replicate desired characteristics of cellular structures in arbitrary amounts of image data [8][9][10][11][12][13] and ideally serve as a way to entirely replace human annotation.…”

Denoising Diffusion Probabilistic Models for Generation of Realistic Fully-Annotated Microscopy Image Data Sets

“…In order to render human annotation efforts obsolete and still be able to generate fully-annotated synthetic image data sets, cellular structures need to be automatically simulated. Multiple sophisticated approaches for automated generation of cellular structures have already been proposed, ranging from physics-based methods [14], statistical shape-models [2,11] and spherical harmonics [11,9], to deep learning-based methods [32,6,33]. In this work we focus on basic approaches by utilizing geometrical functions and use a total of five different fluorescence microscopy image data sets as guidelines for simulation experiments (Fig.…”