Towards automating structural discovery in scanning transmission electron microscopy ^*

Abstract

Scanning transmission electron microscopy (STEM) is now the primary tool for exploring functional materials on the atomic level. Often, features of interest are highly localized in specific regions in the material, such as ferroelectric domain walls, extended defects, or second phase inclusions. Selecting regions to image for structural and chemical discovery via atomically resolved imaging has traditionally proceeded via human operators making semi-informed judgements on sampling locations and parameters. Recent efforts at automation for structural and physical discovery have pointed towards the use of ‘active learning’ methods that utilize Bayesian optimization with surrogate models to quickly find relevant regions of interest. Yet despite the potential importance of this direction, there is a general lack of certainty in selecting relevant control algorithms and how to balance a priori knowledge of the material system with knowledge derived during experimentation. Here we address this gap by developing the automated experiment workflows with several combinations to both illustrate the effects of these choices and demonstrate the tradeoffs associated with each in terms of accuracy, robustness, and susceptibility to hyperparameters for structural discovery. We discuss possible methods to build descriptors using the raw image data and deep learning based semantic segmentation, as well as the implementation of variational autoencoder based representation. Furthermore, each workflow is applied to a range of feature sizes including NiO pillars within a La:SrMnO3 matrix, ferroelectric domains in BiFeO3, and topological defects in graphene. The code developed in this manuscript are open sourced and will be released at github.com/creangnc/AE_Workflows.

Bayesian Active Learning for Scanning Probe Microscopy: From Gaussian Processes to Hypothesis Learning

“…Furthermore, the use of batch updates further necessitates the selection of pathfinder function, i.e., defining the sampling sequence under the constraints of microscope operation, representing a known example of a NP hard problem that we never come back to exactly where we were. Finally, GP will still be expected to require a considerable number of sampling points, and although it can reduce the experimental budget to 10–30% of grid measurements, the trade-off is that it renders the experimental procedure considerably more complex …”

“…Finally, GP will still be expected to require a considerable number of sampling points, and although it can reduce the experimental budget to 10−30% of grid measurements, the trade-off is that it renders the experimental procedure considerably more complex. 86 Overall, the results shown above yield several conclusions. For scanning with nonrectangular trajectories with fixed sampling points, the GP method allows for image reconstruction and offers a less than order of magnitude improvement in sampling.…”

Applications of Transmission Electron Microscopy in Phase Engineering of Nanomaterials

“…The development of an autonomous TEM, capable of automatically and independently executing all of these steps, would greatly enhance the efficiency of materials science research. In recent years, relevant research has commenced, primarily focusing on self-driving microscope, machine learning, or AI-dominated data analysis and image processing. − …”

Optimizing training trajectories in variational autoencoders via latent Bayesian optimization approach *

“…is the distance loss between the prior p (z) distribution (usually chosen as standard Gaussian) and the posterior p(z|x) distributions of the latent representation from data. Different VAE models have been applied to materials systems, in attempt to learn from complex image data [11,38,[42][43][44].…”