An Automated Scanning Transmission Electron Microscope Guided by Sparse Data Analytics

Olszta, Matthew; Hopkins, Derek; Fiedler, Kevin R.; Oostrom, Marjolein; Akers, Sarah; Spurgeon, Steven R.

doi:10.48550/arxiv.2109.14772

Cited by 2 publications

(2 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The ferroelastic walls can be determined on the image directly via human eye; however, finding these automatically is a challenge. Over the last 5 years, deep convolutional neural networks (DCNN) have been broadly adopted in electron [50,51] and scanning probe microscopies. [52][53][54] However, while these techniques have amply demonstrated their potential for postacquisition data analysis, their implementation as a part of real time experiment is highly non-trivial.…”

Section: Resultsmentioning

confidence: 99%

Exploring Physics of Ferroelectric Domain Walls in Real Time: Deep Learning Enabled Scanning Probe Microscopy

et al. 2022

View full text Add to dashboard Cite

The functionality of ferroelastic domain walls in ferroelectric materials is explored in real‐time via the in situ implementation of computer vision algorithms in scanning probe microscopy (SPM) experiment. The robust deep convolutional neural network (DCNN) is implemented based on a deep residual learning framework (Res) and holistically nested edge detection (Hed), and ensembled to minimize the out‐of‐distribution drift effects. The DCNN is implemented for real‐time operations on SPM, converting the data stream into the semantically segmented image of domain walls and the corresponding uncertainty. Further the pre‐defined experimental workflows perform piezoresponse spectroscopy measurement on thus discovered domain walls, and alternating high‐ and low‐polarization dynamic (out‐of‐plane) ferroelastic domain walls in a PbTiO3 (PTO) thin film and high polarization dynamic (out‐of‐plane) at short ferroelastic walls (compared with long ferroelastic walls) in a lead zirconate titanate (PZT) thin film is reported. This work establishes the framework for real‐time DCNN analysis of data streams in scanning probe and other microscopies and highlights the role of out‐of‐distribution effects and strategies to ameliorate them in real time analytics.

show abstract

Section: Resultsmentioning

confidence: 99%

Exploring Physics of Ferroelectric Domain Walls in Real Time: Deep Learning Enabled Scanning Probe Microscopy

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Because PhyDNet is a general architecture for video prediction, and video recording is a common method for material research, we anticipate that PhyDNet can be used on many other scientific videos not limited to the in-situ TEM videos, and the video-prediction capability we developed can be used for a variety of applications. In particular, it may find application in the development of automated experimentation and future data-driven microscope platforms [31,32].…”

Section: Discussionmentioning

confidence: 99%

Deep-Learning-Based Prediction of Nanoparticle Phase Transitions During In Situ Transmission Electron Microscopy

Spurgeon

Wang

et al. 2022

Preprint

View full text Add to dashboard Cite

We develop the machine learning capability to predict a time sequence of in-situ transmission electron microscopy (TEM) video frames based on the combined long-short-term-memory (LSTM) algorithm and the features de-entanglement method. We train deep learning models to predict a sequence of future video frames based on the input of a sequence of previous frames. This unique capability provides insight into size dependent structural changes in Au nanoparticles under dynamic reaction condition using in-situ environmental TEM data, informing models of morphological evolution and catalytic properties. The model performance and achieved accuracy of predictions are desirable based on, for scientific data characteristic, a limited size of training data sets. The model convergence and values for the loss function mean square error show dependence on the training strategy, and structural similarity measure between predicted structure images and ground truth reaches the value of about 0.7. This computed structural similarity is smaller than values obtained when the deep learning architecture is trained using much larger benchmark data sets, it is sufficient to show the structural transition of Au nanoparticles. While performance parameters of our model applied to scientific data fall short of those achieved for the non-scientific big data sets, we demonstrate model ability to predict the evolution, even including the particle structural phase transformation, of Au nano particles as catalyst for CO oxidation under the chemical reaction conditions. Using this approach, it may be possible to anticipate the next steps of a chemical reaction for emerging automated experimentation platforms.

show abstract

An Automated Scanning Transmission Electron Microscope Guided by Sparse Data Analytics

Cited by 2 publications

References 56 publications

Exploring Physics of Ferroelectric Domain Walls in Real Time: Deep Learning Enabled Scanning Probe Microscopy

Exploring Physics of Ferroelectric Domain Walls in Real Time: Deep Learning Enabled Scanning Probe Microscopy

Deep-Learning-Based Prediction of Nanoparticle Phase Transitions During In Situ Transmission Electron Microscopy

Contact Info

Product

Resources

About