Real-time interactive ptychography from electron event representation data

Abstract

The arrival of direct electron detectors (DED) with high frame-rates in the field of scanning transmission electron microscopy has enabled many experimental techniques that require the collection of a full diffraction pattern at each scan position, a field which is subsumed under the name 4D-STEM.DED frame rates approaching 100kHz place stringent requirements on data transmission rates and computing infrastructure. Current commercial DEDs allow the user to make compromises in pixel bit depth, detector binning or windowing to reduce the per-fame file size and allow higher frame rates.

Solving complex nanostructures with ptychographic atomic electron tomography

“…Crop-outs containing only the scan positions covering the Zr 11 Te 50 DW-CNT for all tilts were determined from annular bright-field images (12.5 mrad-25 mrad integration range) of the tilt series for ptychographic reconstruction. Initial guesses for the defocus aberration for each tilt were manually obtained with an interactive real-time implementation 28 of the single-sideband ptychography method 67 . The data in EER format was further preprocessed by centering the position-averaged diffraction pattern, symmetric cropping to a maximum detector angle of 2α, with α the semi-convergence angle, and binning to a detector size of 88 × 88 pixels.…”

“…It can be implemented as a direct phase-retrieval method [22][23][24] or using iterative phase-retrieval algorithms 25,26 using many different experimental configurations 27 . Direct ptychography methods enjoy a low computational complexity and can provide real-time feedback when implemented on modern hardware 28 , while iterative ptychography algorithms allow to jointly solve for the complex sample transmission function 29 , a non-parametric probe wave function 30,31 , sub-pixel scan positions 32,33 , partial coherence effects in the experiment 33,34 , fluctuating illumination during the scan 35 and can provide superior resolution and reconstruction quality under certain conditions 33,36 . Neural-network-based reconstruction methods improve the aforementioned direct methods while retaining real-time reconstruction capabilities 37,38 .…”

A Roadmap for Edge Computing Enabled Automated Multidimensional Transmission Electron Microscopy

“…Ophus et al have demonstrated that the py4DSTEM package can be integrated with microscope data acquisition for automated crystallographic orientation mapping [62]. In another example, it was demonstrated that the obtained 4D-STEM datasets can be used to perform single sideband (SSB) ptychography [63] in real time by using GPUbased solvers on a connected edge system [64,65].…”

Designing better 4D scanning transmission electron microscopy experiments

“…One of the most important hardware improvements of the last few years has been the development of high-speed direct electron detectors capable of capturing diffraction patterns 𝐼𝐼(𝑘𝑘 𝑥𝑥 , 𝑘𝑘 𝑦𝑦 ) as a function of probe position (𝑟𝑟 𝑥𝑥 , 𝑟𝑟 𝑦𝑦 ), yielding a 4-dimensional (4D-STEM) dataset, Figure 1a. [1][2][3][4][5] These spatially resolved diffraction patterns can be used to generate a virtually unlimited number of images by making use of various scattering regimes and more advanced post-processing, such as centre-of-mass imaging, electron ptychography, strain mapping, and virtual dark field, to name a few. [6] State-of-the-art 4D-STEM detectors typically compromise on number of pixels (10 4 to 10 7 pixels), frame rate (10 2 to >10 5 ), and dynamic range (10s to >10 6 electrons), depending on the intended use cases.…”