Event-driven acquisition for content-enriched microscopy

Abstract

In fluorescence microscopy, the amount of information that can be collected from the sample is limited, often due to constraints imposed by photobleaching and phototoxicity. Here, we report an event-driven acquisition (EDA) framework, which combines real-time, neural network-based recognition of events of interest with automated control of the imaging parameters in an instant structured illumination microscope (iSIM). On-the-fly prioritization of imaging rate or experiment duration is achieved by switching between a slow imaging rate to detect the onset of biological events of interest and a fast imaging rate to enable high information content during their progression. In this way, EDA allows the data capture of mitochondrial and bacterial divisions at imaging rates that match their dynamic timescales, while extending the accessible imaging duration, and thereby increases the density of relevant information in the acquired data.

Event-based vision sensor enables fast and dense single-molecule localization microscopy

“…In particular, adapting the processing conditions to the local density of molecules (in space and in time) could improve both the processing time and the localization performance. This is conceptually reminiscent of recent works such as [46], where the size of the spatial range available for PSF shaping is adapted in real time according to the density of molecules (which varies in time during the acquisition), or [47, 48], where the acquisition speed and illumination power are adapted in real time so that it allows recording of data at a sufficient rate while minimizing photo-bleaching, as well as saving storage and processing power. Nevertheless, a fundamental advantage of event-based SMLM is that the sensor records data as fast as it can, which means that no prior knowledge about the acquisition behavior is needed, and that workflow refinements are inherently processing-based.…”

“…More generally, improving the acquisition rate allows more efficient data collection, which can be important to better describe biological phenomena and improve statistical significance. In some cases, it can even be used to develop automated high throughput data collection setups and analysis workflows [32, 33].…”

Multiscale fluorescence imaging of living samples

“…Another strategy that is well suited for multiscale imaging, improving imaging throughput and quality for a region of interest (ROI), is ‘event-driven acquisition (EDA) (Alvelid et al 2021 ; Mahecic et al 2021 ). Unlike the adaptive strategies above that attempt to globally improve image quality, EDA instead attempts to automatically switch between imaging modalities by monitoring real-time changes in the sample (e.g., intensity spikes, local movement, and morphological changes), zooming into the ROI only when necessary.…”

“…This self-driving microscope captures the mitochondrial divisions at imaging rates that match their dynamic time scale, while preserving the sample from unnecessary illumination and extending the accessible imaging duration. a – c were reprinted with permission from ref (Alvelid et al 2021 ), and d – f reprinted with permission from ref (Mahecic et al 2021 ) …”

“…In a second example of EDA (Mahecic et al 2021 ), neural-network based recognition of events of interest was used to automatically control the frame rate of iSIM (Fig. 7 d, e), adjusting it as necessary to capture mitochondrial constriction and division in Cos7 cells (Fig.…”