Richardson–Lucy Deconvolution as a General Tool for Combining Images with Complementary Strengths

Abstract

We use Richardson-Lucy (RL) deconvolution to combine multiple images of a simulated object into a single image in the context of modern fluorescence microscopy techniques. RL deconvolution can merge images with very different point spread functions, such as in multiview light-sheet microscopes [1, 2], while preserving the best resolution information present in each image. We show RL deconvolution is also easily applied to merge high-resolution, high noise images with low-resolution, low noise images, relevant when complementing conventional microscopy with localization microscopy. We also use RL deconvolution to merge images produced via different simulated illumination patterns, relevant to structured illumination microscopy (SIM) [3, 4] and image scanning microscopy (ISM). The quality of our ISM reconstructions are at least as good as standard inversion algorithms for ISM data, but our method follows a simpler recipe that requires no mathematical insight. Finally, we apply RL deconvolution to merge a series of ten images with varying signal and resolution levels. This combination is relevant to gated stimulated-emission depletion (STED) microscopy, and shows that high-quality image merges are possible even in cases where a non-iterative inversion algorithm is unknown.

Two-photon instant structured illumination microscopy improves the depth penetration of super-resolution imaging in thick scattering samples

“…3(a)], even though the images at depth deteriorated relative to those collected at the coverslip surface. In contrast, we were unable to observe recognizable beads at depths >50 μm from the coverslip surface when using 1P excitation (1P ISIM [18]). When setting the illumination conditions for similar SNR at the coverslip, both SNR and the signal-to-background ratio (SBR) degraded much faster with depth when using 1P rather than 2P illumination [Fig.…”

“…By using a single excitation focus in combination with a pinhole, as in image scanning microscopy (ISM) [13] (or multiple point-like excitation foci [14,15] in combination with pinholes) instead of sharp sinusoidal excitation patterns [16], out-of-focus background is inherently rejected by the microscope (i.e., without computation) [see Note S1 in Supplement 1], and depth penetration can be improved to enable 3D super-resolution imaging ~50 μm from the coverslip surface [14]. Despite the apparent differences between sinusoidal SIM and point-based SIM implementations derived from ISM [13], the mechanism of resolution enhancement is the same [17] and, in fact, the same software can be used to process data generated by the two types of SIM [18]. Using multifocal 2P illumination has been shown to further improve SIM performance in thick samples, due to the inherently lower background afforded by 2P excitation [19].…”

A joint Richardson—Lucy deconvolution algorithm for the reconstruction of multifocal structured illumination microscopy data

“…A sophisticated way of processing widefield SIM data was introduced by Ingaramo et al [14] in the form of joint Richardson-Lucy deconvolution, jRL. The algorithm performs the reconstruction without the need of band-separation and repositioning and incorporates an image deconvolution process.…”

“…To reject out-of-focus light and allow optical sectioning we additionally pinhole the raw data by multiplication with Gaussian shaped digital pinholes before starting the actual reconstruction process. Each step of the iteration provides an update of the estimate s x ( ) ⃗ that is a closer approximation of the original structure s x ( ) ⃗ : [14]. Our algorithm, which we subsequently refer to as jRL-MSIM, then combines in a natural fashion a widefield model of MSIM with a deconvolution procedure to reconstruct super-resolved images.…”

Multi-images deconvolution improves signal-to-noise ratio on gated stimulated emission depletion microscopy

“…This strategy was already outlined by Vicidomini et al 7 and Ingaramo et al, 20 but lacks an extensive study.…”