Artificial Scanning Electron Microscopy Images Created by Generative Adversarial Networks from Simulated Particle Assemblies

Abstract

Particle assemblies created by software package Blender are converted into artificial scanning electron micrographs (SEM) with a generative adversarial network (GAN). The introduction of height maps (i.e., surface topography or relief structure) considerably enhances the quality of the artificial SEM images by providing 3D information on the input data. These artificial images serve as input data to train a convolutional neural network (CNN) to identify and classify nanoparticles. Although the performance of the CNN trained with artificial SEM images is slightly inferior to the same CNN trained with real SEM images, this offers a pathway to create training data for segmentation and classification networks for SEM image analysis by deep learning algorithms.

Simulated HRTEM images of nanoparticles to train a neural network to classify nanoparticles for crystallinity

“…As the generation of manually labelled training data for this task is not only time-consuming but also highly error-prone, different image simulation approaches were tested to establish a feasible training pipeline. This follows earlier approaches to train networks with simulated scanning electron microscopy images for particle size analysis, 7,31 created by generative adversarial networks (GANs). 32,33 We present a fully automated classification of nanoparticles by machine learning with respect to their crystallinity, fully based on simulated training data.…”

Automated analysis of transmission electron micrographs of metallic nanoparticles by machine learning

“…The rise of artificial intelligence/machine learning/deep learning has considerably enhanced our ability to train computers to recognize and autonomously analyse particles. Machine learning techniques have already been applied to electron microscopic images where they usually outperform classical image analysis approaches, especially when noisy images or overlapping particles are involved 15,17–25 (see ref. 26–28 for recent reviews).…”