Deep Learning of Atomically Resolved Scanning Transmission Electron Microscopy Images: Chemical Identification and Tracking Local Transformations

Abstract

Abstract:Recent advances in scanning transmission electron and scanning probe microscopies have opened exciting opportunities in probing the materials structural parameters and various functional properties in real space with angstrom-level precision. This progress has been accompanied by an exponential increase in the size and quality of datasets produced by microscopic and spectroscopic experimental techniques. These developments necessitate adequate methods for extracting relevant physical and chemical information from the large datasets, for which a priori information on the structures of various atomic configurations and lattice defects is limited or absent. Here we demonstrate an application of deep neural networks to extract information from atomically resolved images including location of the atomic species and type of defects.We develop a "weakly-supervised" approach that uses information on the coordinates of all atomic species in the image, extracted via a deep neural network, to identify a rich variety of defects that are not part of an initial training set. We further apply our approach to interpret complex atomic and defect transformation, including switching between different coordination of silicon dopants in graphene as a function of time, formation of peculiar silicon dimer with mixed 3-fold and 4-fold coordination, and the motion of molecular "rotor". This deep learning based approach resembles logic of a human operator, but can be scaled leading to significant shift in the way of extracting and analyzing information from raw experimental data. Keywords:STEM, neural networks, weakly-supervised learning, graphene, TMDC. 3In the last decade, the proliferation of electron microscopy and scanning probe microscopy techniques have generated massive amounts of data on local chemical structure and atomic transformation. [1][2][3] Since the advent of aberration corrected Scanning Transmission Electron Microscopy (STEM), atomically resolved images of multiple materials classes ranging from multiferroics, semiconductors, and superconductors have become common. [4][5][6][7][8] The further impetus to this field was given by the development of atomically resolved dynamic studies, when the dynamic changes in matter on the atomic level are visualized. These traditionally include the thermal and chemical processes enabled by advanced thermal and environmental holders. 9,10 More recently, progressively more attention is being attracted to the dynamic processes induced by the electron beam irradiation, 11-14 especially promising in the context of e-beam atomic fabrication. [15][16][17] Similar advances are achieved in the field of atomically resolved scanning tunneling (STM) and atomic force microscopy (AFM). The recent famous examples include direct imaging of chemical bonds in molecules, 18 visualizing atomic collapse in artificial nuclei on graphene, 19 and inferring mechanisms behind fundamental physical phenomena, such as high-T c superconductivity, from single atom defect induced scattering patterns. 20 I...

A Deep Learning Approach to Identify Local Structures in Atomic‐Resolution Transmission Electron Microscopy Images

“…[17] Kirschner and Hillebrand have published a method for predicting defocus and sample thickness, [18] and Meyer and Heindl have used neural networks to reconstruct the exit wave function from off-axis electron holograms. [19] Whereas deep learning methods have recently been proposed to analyze scanning TEM (STEM) images, [20] but have, to our knowledge, not yet been used to analyze the atomic structure in HRTEM images. In this article, we describe a convolutional neural network (CNN) based method for classifying atomic structures in TEM, and demonstrate that it can be applied to single layers of graphene, as well as to supported metallic nanoparticles.…”

A machine perspective of atomic defects in scanning transmission electron microscopy

“…E, Tracking the defect dynamics and symmetry conversion of a Si dopant in a graphene matrix using the neural network. A‐E, Reproduced with permission from Reference Copyright 2017, American Chemical Society…”

“…However, the construction of the simulated images relies on prior knowledge. In the case of point defect identification, Ziatdinov generalized dopants with higher atomic number and vacancies to the ideas of protrusion and depression . The model is only fitted and validated by simulated data on 2D materials.…”

Data-Driven Materials Investigations: The Next Frontier in Understanding and Predicting Fatigue Behavior

“…DeCost and Holm 22 used convolutional neural networks (CNNs) to predict properties of materials samples based on their microstructures. Ziatdinov et al 23 automatically identified defects in atomic-scale microscopy data with deep learning. Xu and LeBeau 24 employed CNNs to dramatically accelerate the process of analyzing electron beam diffraction patterns.…”