Learning surface molecular structures via machine vision

Abstract

Recent advances in high resolution scanning transmission electron and scanning probe microscopies have allowed researchers to perform measurements of materials structural parameters and functional properties in real space with a picometre precision. In many technologically relevant atomic and/or molecular systems, however, the information of interest is distributed spatially in a non-uniform manner and may have a complex multi-dimensional nature. One of the critical issues, therefore, lies in being able to accurately identify ('read out') all the individual building blocks in different atomic/molecular architectures, as well as more complex patterns that these blocks may form, on a scale of hundreds and thousands of individual atomic/molecular units. Here we employ machine vision to read and recognize complex molecular assemblies on surfaces. Specifically, we combine Markov random field model and convolutional neural networks to classify structural and rotational states of all individual building blocks in molecular assembly on the metallic surface visualized in high-resolution scanning tunneling microscopy measurements. We show how the obtained full decoding of the system allows us to directly construct a pair density function-a centerpiece in analysis of disorderproperty relationship paradigm-as well as to analyze spatial correlations between multiple order parameters at the nanoscale, and elucidate reaction pathway involving molecular conformation changes. The method represents a significant shift in our way of analyzing atomic and/or molecular resolved microscopic images and can be applied to variety of other microscopic measurements of structural, electronic, and magnetic orders in different condensed matter systems.

AFLOW-ML: A RESTful API for machine-learning predictions of materials properties

“…Combined with other online databases such as the Materials Project [17], NoMaD [18] and OQMD [19], materials data is abundant and available. As a result, machine learning (ML) methods have emerged as the ideal tool for data analysis [20][21][22][23][24][25], by identifying the key features in a data set [26] to construct a model for predicting the properties of a material. Recently, several models were developed to predict the properties of various material classes such as perovskites [27,28], oxides [29], elpasolites [30,31], thermoelectrics [32][33][34], and metallic glasses [35].…”

Interface Characterization and Control of 2D Materials and Heterostructures

“…Finally, the application of artificial intelligence to materials research is beginning to reshape the discovery of new materials including 2D systems. For example, several developments linking machine learning and data mining with scanning probe image recognition, analysis, and interpretation have been demonstrated . Machine learning assisted material discovery, design, and growth optimization are also gaining momentum.…”

Machine learning at the (sub)atomic scale: next generation scanning probe microscopy

“…[31]; (B)Using a somewhat similar methodology to that employed for (A), Aldritt and co-workers [34] have trained a convolutional neural net to recognise specific molecular geometries in ultrahigh resolution AFM images; (C) Ziatdinov et al determined the orientation/rotation of single molecules on a solid surface via an artificial neural network. In this case the architecture of a convolutional neural network (CNN) is shown but a variety of other machine vision tools are described by Ziatdinov, Maksov, and Kalinin [35].…”

“…Building on previous machine learning protocols developed by Sergei Kalinin and co-workers at Oak Ridge National Laboratories (among others) [29,36,37], Aldritt et al have developed what they describe as automated structure discovery AFM (ASD-AFM), a deep learning framework based on a similar methodology to that of Zhang et al, whereby a large set of simulated images is generated-in this case via a combination of density functional theory (DFT) optimisation of molecular structures and the probe-particle model of tip-sample interactions developed by Hapala et al [38,39] -and a convolutional neural net is used to determine the best match between experimental data and a molecular geometry. Figure 2 illustrates the general CNN methodology adopted by Aldritt et al, alongside Ziatdinov, Maksov, and Kalinin's earlier work on determining the rotational state of adsorbed molecules from STM data [35].…”