A deep learning based automatic defect analysis framework for In-situ TEM ion irradiations

Shen, Mei‐Hua; Li, Guanzhao; Wu, Dongxia; Yaguchi, Yudai; Haley, Jack C.; Field, Kevin G.; Morgan, Dane

doi:10.1016/j.commatsci.2021.110560

Cited by 36 publications

(9 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…MOT is defined as the task of predicting the trajectories of the objects of interest in videos or image sequences. The current tracking application is restricted to dislocation loop tracking 14 and nanoparticle tracking 21 , 22 that use two-compute intensive separate models for object detection and tracking. The tracking model usually contains a traditional computer vision algorithm with a slow rigid feature Re-ID and association method.…”

Section: Introductionmentioning

confidence: 99%

DefectTrack: a deep learning-based multi-object tracking algorithm for quantitative defect analysis of in-situ TEM videos in real-time

Sainju

Chen

Schaefer

et al. 2022

Sci Rep

View full text Add to dashboard Cite

In-situ irradiation transmission electron microscopy (TEM) offers unique insights into the millisecond-timescale post-cascade process, such as the lifetime and thermal stability of defect clusters, vital to the mechanistic understanding of irradiation damage in nuclear materials. Converting in-situ irradiation TEM video data into meaningful information on defect cluster dynamic properties (e.g., lifetime) has become the major technical bottleneck. Here, we present a solution called the DefectTrack, the first dedicated deep learning-based one-shot multi-object tracking (MOT) model capable of tracking cascade-induced defect clusters in in-situ TEM videos in real-time. DefectTrack has achieved a Multi-Object Tracking Accuracy (MOTA) of 66.43% and a Mostly Tracked (MT) of 67.81% on the test set, which are comparable to state-of-the-art MOT algorithms. We discuss the MOT framework, model selection, training, and evaluation strategies for in-situ TEM applications. Further, we compare the DefectTrack with four human experts in quantifying defect cluster lifetime distributions using statistical tests and discuss the relationship between the material science domain metrics and MOT metrics. Our statistical evaluations on the defect lifetime distribution suggest that the DefectTrack outperforms human experts in accuracy and speed.

show abstract

Section: Introductionmentioning

confidence: 99%

DefectTrack: a deep learning-based multi-object tracking algorithm for quantitative defect analysis of in-situ TEM videos in real-time

Sainju

Chen

Schaefer

et al. 2022

Sci Rep

View full text Add to dashboard Cite

show abstract

“…Particularly, the YOLO convolutional neural network has been implemented previously with the purpose of detecting different structures in TEM images, not only particles such as cavities or other deformations on the surface of a specimen that has been radiated with an ion beam [ 10 ]. Because YOLO excels at real time detection, it has been used for in situ detection and tracking of objects found in a video feed from a transmission electron microscope [ 11 ]. It can also be used in tandem with other CNN architectures to obtain an improved instance segmentation model in one-dimensional nanostructures such as

nanowire arrays [ 12 ].…”

Section: Introductionmentioning

confidence: 99%

Nanoparticle Detection on SEM Images Using a Neural Network and Semi-Synthetic Training Data

2022

View full text Add to dashboard Cite

Processing images represents a necessary step in the process of analysing the information gathered about nanoparticles after characteristic material samples have been scanned with electron microscopy, which often requires the use of image processing techniques or general purpose image manipulation software to carry out tasks such as nanoparticle detection and measurement. In recent years, the use of networks has been successfully implemented to detect and classify electron microscopy images as well as the objects within them. In this work, we present four detection models using two versions of the YOLO neural network architectures trained to detect cubical and quasi-spherical particles in SEM images; the training datasets are a mixture of real images and synthetic ones generated by a semi-arbitrary method. The resulting models were capable of detecting nanoparticles in images different than the ones used for training and identifying them in some cases as the close proximity between nanoparticles proved a challenge for the neural networks in most situations.

show abstract

“…Roberts et al developed a network called DefectSegNet to classify crystallographic defects in the diffraction contrast STEM images of structural alloy (Roberts et al 2019), and Lee et al trained a network to locate and identify defects in averaged high-resolution STEM images and to enable strain mapping (Lee et al 2020). Other types of NN have also been used to segment STEM images and classify crystallographic defects, including Faster Regional Convolutions Neural Network (Faster RCNNs) (Shen, Li, Wu, Liu, et al 2021), Mask RCNN (Jacobs et al 2022), and You Only Look Once (YOLO) (Shen, Li, Wu, Yaguchi, et al 2021).…”

Section: Introductionmentioning

confidence: 99%

Benchmark tests of atom-locating CNN models with a consistent dataset

et al. 2021

Self Cite

View full text Add to dashboard Cite

A deep learning based automatic defect analysis framework for In-situ TEM ion irradiations

Cited by 36 publications

References 59 publications

DefectTrack: a deep learning-based multi-object tracking algorithm for quantitative defect analysis of in-situ TEM videos in real-time

DefectTrack: a deep learning-based multi-object tracking algorithm for quantitative defect analysis of in-situ TEM videos in real-time

Nanoparticle Detection on SEM Images Using a Neural Network and Semi-Synthetic Training Data

Benchmark tests of atom-locating CNN models with a consistent dataset

Contact Info

Product

Resources

About