Event-based vision sensor enables fast and dense single-molecule localization microscopy

Abstract

Single-molecule localization microscopy (SMLM) applications are often hampered by the fixed frame rate at which data acquisition is performed. Here, we present an alternative new approach of acquiring and processing SMLM data based on an event-based (or neuromorphic vision) sensor. This type of sensor reacts to light intensity changes rather than integrating the number of photons during each frame exposure time. This makes them particularly suited to SMLM, where the ability to surpass the diffraction-limited resolution is provided by blinking events. Each pixel works independently and returns a signal only when an intensity change is detected; intensity changes are returned as a list with pixel positions and timestamps rather than frames with a fixed frame rate. Since the output is a sparse list containing only useful data (for example a molecule turning on or off), the temporal resolution is significantly faster than typical speeds of EMCCD and sCMOS cameras. Here we demonstrate the feasibility of SMLM super-resolution imaging with this type of event-based sensors which, in addition, are more affordable than EMCCD or sCMOS cameras. We characterize the localization precision and show that it is equivalent to that of frame-based scientific cameras, including on fluorescently labelled biological samples. Furthermore, taking advantage of the unique properties of the sensor, we use event-based SMLM to perform very dense single-molecule acquisitions, where frame-based cameras experience significant limitations. All the data processing codes are made available.

Computational neuromorphic imaging: principles and applications

“…Therefore, compared with conventional microscopic methods, event sensors that only respond to local changes in light intensity can enable weak signal detection and super-resolution via the localization microscopy with a higher sensitivity. 7,16 The CNI-enhanced super-resolution approach has widespread applications in precisely tracking and understanding stochastic processes in biomedical engineering. Moreover, many super-resolution imaging techniques are exploiting machine learning to provide rapid analysis of event-stream datasets.…”

“…Over the past decade, vast amounts of vision applications have been implemented based on event sensors. 5 Recently, in pursuit of superior imaging performance, many researchers have also used it in the field of optical applications, e.g., super-resolution imaging in localization microscopy 7 and speckle analysis in lensless imaging systems. 8 The motivation for using event sensors over conventional frame-based image sensors lies in their superior capability to process super-fast dynamics and ultra-high dynamic range of scenes.…”

Latest trends in bioimaging and building a proactive network of early-career young scientists around bioimaging in Europe

“…The scientific program of our Third Imabio YSN conference was divided into three different sessions. Within the session dedicated to the Bioinformatics and Optics, Clément Cabriel (ESPCI, Paris, France) talked about high density single molecule localization using event-based sensors, which is an alternative new approach of acquiring and processing single-molecule localization microscopy (SMLM) data based on an event-based (or neuromorphic vision) sensor ( Cabriel et al, 2022 preprint). His talk was followed by a presentation from Jiachuan Bai (Institut Pasteur, Paris, France), on ShareLoc – an open platform for sharing localization microscopy data ( Ouyang et al, 2022 ).…”