Compressed Sensing and Electron Microscopy

Binev, Peter; Dahmen, Wolfgang; DeVore, Ronald A.; Lamby, Philipp; Savu, Daniel; Sharpley, Robert

doi:10.1007/978-1-4614-2191-7_4

Cited by 48 publications

(33 citation statements)

References 33 publications

(46 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…5,6 The direct acquisition of sparse images presents the important advantage of reducing radiation dose, acquisition time and data size. 6,7 An easy and implementable choice for sparse sampling is a fully random distribution of the pixels, which permit high subsampling with minimal reconstruction distortions. 8 The applicability of different reconstruction algorithms for S(T)EM imaging has been demonstrated on simulated random scan images obtained by extracting a subset of randomly chosen pixels from full images.…”

Section: Introductionmentioning

confidence: 99%

Spatial and spectral dynamics in STEM hyperspectral imaging using random scan patterns

et al. 2020

View full text Add to dashboard Cite

The evolution of the scanning modules for scanning transmission electron microscopes (STEM) has realized the possibility to generate arbitrary scan pathways, an approach currently explored to improve acquisition speed and to reduce electron dose effects. In this work, we present the implementation of a random scan operating mode in STEM achieved at the hardware level via a custom scan control module. A pre-defined pattern with fully shuffled raster order is used to sample the entire region of interest. Subsampled random sparse images can then be extracted at successive time frames, to which suitable image reconstruction techniques can be applied. With respect to the conventional raster scan mode, this method permits to limit dose accumulation effects, but also to decouple the spatial and temporal information in hyperspectral images. We provide some proofs of concept of the flexibility of the random scan operating mode, presenting examples of its applications in different spectro-microscopy contexts: atomically-resolved elemental maps with electron energy loss spectroscopy and nanoscale-cathodoluminescence spectrum images. By employing adapted post-processing tools, it is demonstrated that the method allows to precisely track and correct for sample instabilities and to follow spectral diffusion with a high spatial localization. arXiv:1909.07842v1 [physics.app-ph]

show abstract

Section: Introductionmentioning

confidence: 99%

Spatial and spectral dynamics in STEM hyperspectral imaging using random scan patterns

et al. 2020

View full text Add to dashboard Cite

show abstract

“…For many samples, however, including many biological specimens, lowering the acceleration voltage does not prevent damage as radiolysis is the dominant mechanism. Although a number of lowdose high resolution (HR) schemes have been explored with some level of success [1], the promise held by so-called compressive sensing (CS) approaches is yet to be fully exploited [2].Typically, the Nyquist frequency determines the sampling rate required to resolve a specific feature in a dataset and as such provides a lower limit for the electron dose necessary to acquire the data experimentally. In practice this requires oversampling by twice the maximum relevant frequency.…”

mentioning

confidence: 99%

mentioning

confidence: 99%

Practical Implementation of Compressive Sensing for High Resolution STEM

et al. 2016

View full text Add to dashboard Cite

New generation scanning transmission electron microscopes (STEMs) allow for atomic resolution imaging and spectroscopy at low beam energy, in many cases below the threshold for primary knock-on damage. For many samples, however, including many biological specimens, lowering the acceleration voltage does not prevent damage as radiolysis is the dominant mechanism. Although a number of lowdose high resolution (HR) schemes have been explored with some level of success [1], the promise held by so-called compressive sensing (CS) approaches is yet to be fully exploited [2].Typically, the Nyquist frequency determines the sampling rate required to resolve a specific feature in a dataset and as such provides a lower limit for the electron dose necessary to acquire the data experimentally. In practice this requires oversampling by twice the maximum relevant frequency. However, by applying the CS principles of sparsity and incoherence this limit can be overcome. CS theory then allows to recover the essential image information from randomly subsampled images. The theory of CS has been successfully applied in a number of areas, e.g. astronomy, MRI scanning and electron tomography [3]. As STEM, both imaging [4] and spectroscopy, is a controllable scanning probe technique it lends itself to the CS approach. Despite this huge potential, CS acquisition schemes able to recover quantitative information have so far not been implemented for scanning transmission electron microscopy.Here we demonstrate such a CS-based STEM acquisition implementation and benchmark its performance at atomic-resolution using a variety of systems, including as illustrated in fig. 1, complex oxide ceramics with an inhomogeneous distribution of point defects, but also highly beam-sensitive materials for catalysis with atomic resolution [5]. The implementation relies on custom-built hardware and software which control an electrostatic beam shutter to blank the electron beam during all but a few randomly chosen pixels through a regular image (or indeed spectrum image) acquisition. The total electron dose used to form a whole dataset can thus be drastically reduced whilst maintaining all other acquisition parameters identical to regular data acquisition, allowing a seamless transition from 'fully sampled' to ultra-low-dose conditions. We show that datasets acquired in this fashion down to a few % of the total incoming dose can be reconstructed to yield a truthful atomic resolution representation of the sample, using a variety of reconstruction algorithms [4,6]. We show that this implementation of CS can also be extended to 2D spectroscopy mapping and simultaneous HAADF image acquisition, see fig. 2. These results clearly open the door to practical ultra-low dose high-resolution imaging and spectroscopy in the STEM for very beam sensitive samples, where the level of sampling can simply be chosen depending on the damage threshold of the material being investigated [7].

show abstract

“…Since the publication of Candès, there has been enormous growth in the application of CS and development of CS variants. For electron microscopy applications, the concept of CS has also been recently applied to electron tomography [6], and reduction of electron dose in scanning transmission electron microscopy (STEM) imaging [7].To demonstrate the applicability of coded aperture CS video reconstruction for atomic level imaging, we simulate compressive sensing on observations of Pd nanoparticles and Ag nanoparticles during exposure to high temperatures and other environmental conditions. Figure 1 highlights the results from the Pd nanoparticle experiment.…”

mentioning

confidence: 99%

mentioning

confidence: 99%

TEM Video Compressive Sensing

Stevens¹,

Kovařík²,

Abellán³

et al. 2015

Microsc Microanal

View full text Add to dashboard Cite

One of the main limitations of imaging at high spatial and temporal resolution during in-situ TEM experiments is the frame rate of the camera being used to image the dynamic process. While the recent development of direct detectors has provided the hardware to achieve frame rates approaching 0.1ms, the cameras are expensive and must replace existing detectors. In this paper, we examine the use of coded aperture compressive sensing methods [1,2,3,4] to increase the framerate of any camera with simple, low-cost hardware modifications. The coded aperture approach allows multiple sub-frames to be coded and integrated into a single camera frame during the acquisition process, and then extracted upon readout using statistical compressive sensing inversion. Our simulations show that it should be possible to increase the speed of any camera by at least an order of magnitude.Compressive Sensing (CS) combines sensing and compression in one operation, and thus provides an approach that could further improve the temporal resolution while correspondingly reducing the electron dose rate. Because the signal is measured in a compressive manner, fewer total measurements are required. When applied to TEM video capture, compressive imaging could improve acquisition speed and reduce the electron dose rate. CS is a recent concept, and has come to the forefront due the seminal work of Candès [5]. Since the publication of Candès, there has been enormous growth in the application of CS and development of CS variants. For electron microscopy applications, the concept of CS has also been recently applied to electron tomography [6], and reduction of electron dose in scanning transmission electron microscopy (STEM) imaging [7].To demonstrate the applicability of coded aperture CS video reconstruction for atomic level imaging, we simulate compressive sensing on observations of Pd nanoparticles and Ag nanoparticles during exposure to high temperatures and other environmental conditions. Figure 1 highlights the results from the Pd nanoparticle experiment. On the left, 10 frames are reconstructed from a single coded frame-the original frames are shown for comparison. On the right a selection of three frames are shown from reconstructions at compression levels 10, 20, 30. The reconstructions, which are not post-processed, are true to the original and degrade in a straightforward manner. The final choice of compression level will obviously depend on both the temporal and spatial resolution required for a specific imaging task, but the results indicate that an increase in speed of better than an order of magnitude should be possible for all experiments [8].References:[1] P Llull, X Liao, X Yuan et al.

show abstract

Compressed Sensing and Electron Microscopy

Cited by 48 publications

References 33 publications

Spatial and spectral dynamics in STEM hyperspectral imaging using random scan patterns

Spatial and spectral dynamics in STEM hyperspectral imaging using random scan patterns

Practical Implementation of Compressive Sensing for High Resolution STEM

TEM Video Compressive Sensing

Contact Info

Product

Resources

About