Development of AI-assisted microscopy frameworks through realistic simulation in pySTED

Abstract

The integration of artificial intelligence into microscopy systems significantly enhances performance, optimizing both the image acquisition and analysis phases. Development of artificial intelligence (AI)-assisted super-resolution microscopy is often limited by the access to large biological datasets, as well as by the difficulties to benchmark and compare approaches on heterogeneous samples. We demonstrate the benefits of a realistic STED simulation platform, pysted, for the development and deployment of AI-strategies for super-resolution microscopy. The simulation environment provided by pysted allows the augmentation of data for the training of deep neural networks, the development of online optimization strategies, and the training of reinforcement learning models, that can be deployed successfully on a real microscope.