The role of convolutional neural networks in scanning probe microscopy: a review

Abstract

Progress in computing capabilities has enhanced science in many ways. In recent years, various branches of machine learning have been the key facilitators in forging new paths, ranging from categorizing big data to instrumental control, from materials design through image analysis. Deep learning has the ability to identify abstract characteristics embedded within a data set, subsequently using that association to categorize, identify, and isolate subsets of the data. Scanning probe microscopy measures multimodal surface properties, combining morphology with electronic, mechanical, and other characteristics. In this review, we focus on a subset of deep learning algorithms, that is, convolutional neural networks, and how it is transforming the acquisition and analysis of scanning probe data.

Electrostatic Discovery Atomic Force Microscopy

“…In general, the adoption of ML methods into materials analysis has seen rapid recent growth, − and this has been followed by an equivalent growth in its applications to image analysis in SPM. ,− Here, we build upon our ML method for predicting molecular structure from AFM images, to predict the electrostatic field of the sample molecule.…”

Bayesian Active Learning for Scanning Probe Microscopy: From Gaussian Processes to Hypothesis Learning

“…This has generated much enthusiasm on combining machine-learning-based data analysis within the microscope data acquisition loop, i.e., the concept of automated and autonomous microscopy. Correspondingly, a number of opinion and analysis pieces have been published over the past few years. ,,,− …”

A Robust Neural Network for Extracting Dynamics from Electrostatic Force Microscopy Data

“…NNs are sensitive to biases introduced through the training process, such as skewed or unrealistic distributions of training data leading to poor performance on test data. ,,,, To combat such bias, we simulated a training data set with target τ values ranging from 1 to 500 μs with a precision of 10 ns. We used uniform distribution of the log (base 10) of these target values so that the network had a uniform representation of the 3 orders of magnitude (see Figure S3b).…”

“…In practice, the use of ML for SPM data processing provides many potential advantages, namely, robustness against noise, speed in application, and ease of handling multidimensional data. ,,,− , Here, we demonstrate the use of a neural network (NN) to process data-rich SPM images from a feedback-free implementation of trEFM (Figure a). , trEFM has the ability to extract transient dynamics down to the ∼10 ns regime and has been used to study dynamics in systems ranging from organic solar cells ,− to perovskite semiconductors. , In this method, a transient perturbation whose characteristic timescale is governed by the cantilever’s local dynamic properties induces a change in the SPM cantilever’s motion. The exact time-dependent profile of the sample’s response to this excitation alters the transient motion of the SPM cantilever, thus encoding details of the short time perturbation that can be discerned by analyzing the evolution of the cantilever at subsequent times. ,,, Previously, we have recovered the time dynamics using a straightforward calibration procedure (Note S1).…”