Real-Time Interactive 4D-STEM Phase-Contrast Imaging From Electron Event Representation Data: Less computation with the right representation

Abstract

The arrival of direct electron detectors (DED) with high frame-rates in the field of scanning transmission electron microscopy has enabled many experimental techniques that require collection of a full diffraction pattern at each scan position, a field which is subsumed under the name four dimensional-scanning transmission electron microscopy (4D-STEM). DED frame rates approaching 100 kHz require data transmission rates and data storage capabilities that exceed commonly available computing infrastructure. Current commercial DEDs allow the user to make compromises in pixel bit depth, detector binning or windowing to reduce the per-frame file size and allow higher frame rates. This change in detector specifications requires decisions to be made before data acquisition that may reduce or lose information that could have been advantageous during data analysis. The 4D Camera, a DED with 87 kHz frame-rate developed at Lawrence Berkeley National Laboratory, reduces the raw data to a linear-index encoded electron event representation (EER). Here we show with experimental data from the 4D Camera that linearindex encoded EER and its direct use in 4D-STEM phase contrast imaging methods enables real-time, interactive phase-contrast from large-area 4D-STEM datasets. We detail the computational complexity advantages of the EER and the necessary computational steps to achieve real-time interactive ptychography and center-of-mass differential phase contrast using commonly available hardware accelerators.

Three-dimensional electron ptychography of organic–inorganic hybrid nanostructures

“…If raw data volumes in a dense representation are considered then there is also a substantial increase in data volume with field of view for this technique. However, we note that there have been reports of advanced data processing strategies to mitigate this 68,90 . In contrast, defocus-probe ptychography can be preformed with relatively simple detectors and requires less raw data volume for an object of the same size.…”

Solving Complex Nanostructures With Ptychographic Atomic Electron Tomography

“…Crop-outs containing only the scan positions covering the Zr 11 Te 50 DW-CNT for all tilts were determined from the manually aligned annular bright-field images of the tilt series for ptychographic reconstruction. Initial guesses for the defocus aberration for each tilt were manually obtained with an interactive real-time implementation [36] of the single-sideband ptychography (SSB) method [37].…”

Real-Time Integration Center of Mass (riCOM) Reconstruction for 4D STEM

“…Compared with iterative processes, they are less computationally demanding and guarantee unique solutions since they do not depend on optimization algorithms. Also, some prior knowledge, such as the prediction of a phase distribution that may arise from astigmatism and defocus, can be provided to this post process for acquiring more detailed information (Pelz et al, 2021). However, the ability to achieve fast reconstructions, regardless whether they are iterative or not, usually relies on accelerators (e.g., GPUs), as well as large amounts of computer memory in order to accommodate the whole dataset, or some reduced version of it.…”