Generation of microbial colonies dataset with deep learning style transfer

Abstract

We introduce an effective strategy to generate an annotated synthetic dataset of microbiological images of Petri dishes that can be used to train deep learning models in a fully supervised fashion. The developed generator employs traditional computer vision algorithms together with a neural style transfer method for data augmentation. We show that the method is able to synthesize a dataset of realistic looking images that can be used to train a neural network model capable of localising, segmenting, and classifying five different microbial species. Our method requires significantly fewer resources to obtain a useful dataset than collecting and labeling a whole large set of real images with annotations. We show that starting with only 100 real images, we can generate data to train a detector that achieves comparable results (detection mAP $$=0.416$$ = 0.416 , and counting MAE $$=4.49$$ = 4.49 ) to the same detector but trained on a real, several dozen times bigger dataset (mAP $$=0.520$$ = 0.520 , MAE $$=4.31$$ = 4.31 ), containing over 7 k images. We prove the usefulness of the method in microbe detection and segmentation, but we expect that it is general and flexible and can also be applicable in other domains of science and industry to detect various objects.

Self-normalized density map (SNDM) for counting microbiological objects

“…We close the discussion of the numerical result by comparing the SNDM model predictions with the other popular counting systems. In one of our previous papers 23 , 25 we discussed some of them. For the comparison, we choose two of the most successful approaches (in our experiments) to count AGAR microbiological colonies, namely, Faster R-CNN detector 37 and Cascade R-CNN detector 38 .…”

“…The biggest advantage of this method is that it is enough to label each image with the number of objects in it. If a dataset includes more detailed annotation with a bounding box for every object in an image, it is feasible to leverage detectors for object counting 23 – 25 . Eventually, an autoencoder can be used to estimate the density map (DM) based on a given image, which can be later integrated to obtain the number of objects 26 – 30 .…”

“…Traditionally, this task is done manually or semi-automatically using traditional computer vision methods 31 , 32 . However, recent studies prove that a DL-based methodology accelerates the process 23 , 25 , 29 , 33 – 35 . We shall utilize the DM method to predict the number of microbial colonies on a Petri dish.…”

Video frame prediction of microbial growth with a recurrent neural network

“…This emphasizes the need to create large databases of microbial growth, which could be accomplished by combining microfluidics and fluorescence microscopy. A recent study, however, offers a new possibility for increasing the dataset: In Pawlowski et al ( 2022 ) used deep learning style transfer to create a dataset of synthetic images of realistic microbial growth. This technique could be useful for augmenting the dataset for video frame prediction.…”

Semi-Automatic 3D Reconstruction of Atheroma Plaques from Intravascular Ultrasound Images Using an ad-hoc Algorithm

“…Numerous advances in biomedicine have been developed in recent decades. Within the medical context, these techniques have been also successfully applied to different clinical applications, for instance; diagnosis of cancer or melanomas [1][2][3], personalization of drug doses [4], and generation of synthetic dataset of microbial colonies [5]. They have been also used in the cardiovascular field, for instance; treating cardiovascular diseases [6,7], cardiovascular tissue characterisation [8,9] and for the interpretation of electrocardiography signals [10].…”