Cellcyclegan: Spatiotemporal Microscopy Image Synthesis Of Cell Populations Using Statistical Shape Models And Conditional Gans

Abstract

Automatic analysis of spatio-temporal microscopy images is inevitable for state-of-the-art research in the life sciences. Recent developments in deep learning provide powerful tools for automatic analyses of such image data, but heavily depend on the amount and quality of provided training data to perform well. To this end, we developed a new method for realistic generation of synthetic 2D+t microscopy image data of fluorescently labeled cellular nuclei. The method combines spatiotemporal statistical shape models of different cell cycle stages with a conditional GAN to generate time series of cell populations and provides instance-level control of cell cycle stage and the fluorescence intensity of generated cells. We show the effect of the GAN conditioning and create a set of synthetic images that can be readily used for training and benchmarking of cell segmentation and tracking approaches.

Denoising Diffusion Probabilistic Models for Generation of Realistic Fully-Annotated Microscopy Image Data Sets

“…In order to render human annotation efforts obsolete and still be able to generate fully-annotated synthetic image data sets, cellular structures need to be automatically simulated. Multiple sophisticated approaches for automated generation of cellular structures have already been proposed, ranging from physics-based methods [14], statistical shape-models [2,11] and spherical harmonics [11,9], to deep learning-based methods [32,6,33]. In this work we focus on basic approaches by utilizing geometrical functions and use a total of five different fluorescence microscopy image data sets as guidelines for simulation experiments (Fig.…”

“…Deep learning-based approaches proved to be able to become generalist and robust [27,17,20], but training those approaches is still hampered by the scarcity of publicly available annotated image data sets. In order to overcome this issue, either strong augmentations can be introduced to the image data at arbitrary points within the processing pipeline [36,26,12,35] or synthetic image data can be generated to enrich existing data sets [11,2,4,31,28,30].…”

Denoising diffusion probabilistic models for generation of realistic fully-annotated microscopy image datasets

“…Annotation efforts are reduced by automated data augmentation approaches [3][4][5] and tweaked segmentation pipelines [6,7], which help to ease the challenge, but still demand a small set of fully-annotated image data as a basis. Alternatively, automated simulation approaches replicate desired characteristics of cellular structures in arbitrary amounts of image data [8][9][10][11][12][13] and ideally serve as a way to entirely replace human annotation.…”

Implicit Neural Representations for Generative Modeling of Living Cell Shapes

“…This fuels an interest in methods for synthesis of microscopy images and accompanying ground truth masks. The synthesis of microscopy images from ground truth masks has been widely studied [5], and has taken a major leap with the advent of generative adversarial networks [6][7][8][9][10][11][12]. In this work, we focus on the synthesis of ground truth masks.…”

“…Here, a key question is how cell shapes should be represented. A range of parametric models have been proposed that use ellipses [6] or elliptical Fourier descriptors [13] in 2D, statistical shape models [8] in 2D+time, ellipsoids [11,12] (3D) and spherical harmonics [14] in 3D, or ellipsoids deformed using active contours in 3D+time [5]. Deep learning has led to the popularization of volumetric voxel-based representations in 3D due to their natural integration with CNN architectures [9,10,15].…”