Virtual High-Framerate Microscopy Of The Beating Heart Via Sorting Of Still Images

Abstract

Microscopy of the beating heart in embryos provides key insights for the study of its development. However, achieving a sufficiently high framerate is difficult with conventional cameras. Here, we present a method to reconstruct an image sequence covering one heartbeat from images acquired over multiple cardiac cycles, with each image triggered at an arbitrary time, by sorting them according to their similarity. We formulate this task as a traveling salesman problem for which efficient solutions are available. We characterized our approach by evaluating its accuracy on synthetically generated data and sub-sampled high-speed movies of the beating heart in zebrafish larvae. We found that reconstructions are reliable when each phase produces a distinct image and when there are no abrupt cardiac motions, which amounts to collecting at least 100 images in a typical microscopy imaging scenario. We finally demonstrate that our method can be applied on data acquired with a fast confocal microscope, increasing its limited frame-rate by a factor 8.

Reconstruction of Image Sequences From Ungated and Scanning-Aberrated Laser Scanning Microscopy Images of the Beating Heart

“…We further assume that the sorted phases are equally spaced. In [27], we observed that the discrete signal f [k, , n] (Eq. (2)) had to be asymmetric for the sorting to be successful.…”

“…1(a) and (b)) so that the starting times t n are in order of increasing cardiac phase (see Section III-A, Fig 1(c)); next, we estimate the duration T of one heartbeat (see Section III-B, Fig 1(d)) such that the pixels in the sorted frames can then be assigned to their position in the cardiac phase space; finally, we resample the phase-ordered scanning-aberrated data on a uniform grid to obtain the final estimatef (see Section III-C, Fig 1(e)). Note that, in [27], only the special case where the vertical scanning rate is instantaneous was considered, i.e. s = 0, and the problem therefore limited to sorting frames.…”

“…Tralie and Berger [26] created high temporal resolution movies of periodic subjects by estimating the period of the signal then the phases within a sliding window. We recently proposed a cardiac phase sorting method (based on a traveling salesman problem (TSP) formulation) to recover the cardiac phase of a series of still images of a beating heart [27]. A similar TSP approach, albeit using different distance metrics and boundary conditions, has been proposed by Hanslovsky et al [28] to sort out-of-order electron microscopy image stacks.…”

“…We formulate the image reconstruction problem as a combined sorting and space-time unshearing problem, whose We propose to solve this image reconstruction problem by splitting it into two variational problems to recover the unknown parameters, followed by data resampling. Specifically, we cast the problem of retrieving the images' unknown phases as a shortest path search problem with periodic boundary conditions to match the cyclical nature of the cardiac heartbeat and with a pixel-wise similarity criterion as a distance metric, an approach whose efficacy we already demonstrated for sorting unaberrated cardiac images [27]. Following sorting we formulate the second problem as a spacetime shearing problem whose unknown spacing parameter we recognize to coincide with the unknown cardiac period.…”

“…We follow the phase-sorting method described in [27], and define the cardiac phases φ n of the n th frame g[:, :, n], as the wrapping operation…”