Exit Wavefunction Reconstruction from Single Transmission Electron Micrographs with Deep Learning

Abstract

Half of wavefunction information is undetected by conventional transmission electron microscopy (CTEM) as only the intensity, and not the phase, of an image is recorded. Following successful applications of deep learning to optical hologram phase recovery, we have developed neural networks to recover phases from CTEM intensities for new datasets containing 98340 exit wavefunctions. Wavefunctions were simulated with clTEM multislice propagation for 12789 materials from the Crystallography Open Database. Our networks can recover 224×224 wavefunctions in ∼25 ms for a large range of physical hyperparameters and materials, and we demonstrate that performance improves as the distribution of wavefunctions is restricted. Phase recovery with deep learning overcomes the limitations of traditional methods: it is live, not susceptible to distortions, does not require microscope modification or multiple images, and can be applied to any imaging regime. This paper introduces multiple approaches to CTEM phase recovery with deep learning, and is intended to establish starting points to be improved upon by future research. Source code and links to our new datasets and pre-trained models are available at https://github.com/Jeffrey-Ede/one-shot.

Warwick Electron Microscopy Datasets

“…We simulated 98340 TEM exit wavefunctions to train ANNs to reconstruct phases from amplitudes 3 . Half of wavefunction information is undetected by conventional TEM as only the amplitude, and not the phase, of an image is recorded.…”

“…Visualization of complex exit wavefunctions is complicated by the display of their real and imaginary components. However, real and imaginary components are related 3 and can be visualized in the same image by plotting them in red and blue colour channels, respectively. Distributions of 96×96 simulated wavefunctions are shown in figure 4, figure 5, and figure 6 for a large range of materials and physical hyperparameters, a large range of materials and a small range of physical hyperparameters, and In 1.7 K 2 Se 8 Sn 2.28 and a large range of physical hyperparameters, respectively.…”

“…We have set up a new dataserver 1 to make our large new electron microscopy datasets available to both electron microscopists and the wider community. There are three main datasets containing 19769 experimental scanning transmission electron microscopy 2 (STEM) images, 17266 experimental transmission electron microscopy 2 (TEM) images and 98340 simulated TEM exit wavefunctions 3 . Experimental datasets represent general research and were collected by dozens of University of Warwick scientists working on hundreds of projects between January 2010 and June 2018.…”

“…Experimental datasets represent general research and were collected by dozens of University of Warwick scientists working on hundreds of projects between January 2010 and June 2018. We have been using our datasets to train artificial neural networks (ANNs) for electron microscopy [3][4][5][6][7] , where standardizing results with common test sets has been essential for comparison. This paper provides details of and visualizations for datasets and their variants, and is supplemented by source code for analysis, data collection, and both static and interactive visualizations 8 .…”

Review: Deep Learning in Electron Microscopy

“…Non-iterative methods based on DNNs have been developed to reconstruct optical exit wavefunctions from focal series 69 or single images [365][366][367] . Following, DNNs have been developed to reconstruct exit wavefunctions from single TEM images 348 , as shown in figure 4. Indeed, deep learning is increasingly being applied to accelerated quantum mechanics [368][369][370][371][372][373] .…”