Enabling autonomous scanning probe microscopy imaging of single molecules with deep learning

Sotres, Javier; Boyd, Hannah; Gonzalez-Martinez, Juan Francisco

doi:10.1039/d1nr01109j

Cited by 30 publications

(27 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This framework can be extended further with ML methods incorporated into the workflow. An example of this approach can be the recent work of applying a deep convolutional neural network-based YOLOv3 (You Only Look Once) system as an object detector to find the locations of molecules in AFM images, subsequently allowing acquiring high-resolution images of selected molecules . Meanwhile, a Siamese neural network is applied to identify the same molecule in AFM images, tracking the imaged molecules and subsequently avoiding duplicated imaging of the same molecule .…”

Section: Resultsmentioning

confidence: 99%

Bayesian Active Learning for Scanning Probe Microscopy: From Gaussian Processes to Hypothesis Learning

et al. 2022

View full text Add to dashboard Cite

Recent progress in machine learning methods and the emerging availability of programmable interfaces for scanning probe microscopes (SPMs) have propelled automated and autonomous microscopies to the forefront of attention of the scientific community. However, enabling automated microscopy requires the development of task-specific machine learning methods, understanding the interplay between physics discovery and machine learning, and fully defined discovery workflows. This, in turn, requires balancing the physical intuition and prior knowledge of the domain scientist with rewards that define experimental goals and machine learning algorithms that can translate these to specific experimental protocols. Here, we discuss the basic principles of Bayesian active learning and illustrate its applications for SPM. We progress from the Gaussian process as a simple data-driven method and Bayesian inference for physical models as an extension of physics-based functional fits to more complex deep kernel learning methods, structured Gaussian processes, and hypothesis learning. These frameworks allow for the use of prior data, the discovery of specific functionalities as encoded in spectral data, and exploration of physical laws manifesting during the experiment. The discussed framework can be universally applied to all techniques combining imaging and spectroscopy, SPM methods, nanoindentation, electron microscopy and spectroscopy, and chemical imaging methods and can be particularly impactful for destructive or irreversible measurements.

show abstract

Section: Resultsmentioning

confidence: 99%

Bayesian Active Learning for Scanning Probe Microscopy: From Gaussian Processes to Hypothesis Learning

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Here, we develop a novel method for predicting the tipsample interaction forces of dynamic AFM by making use of the recent advances in data science 30,31 and machine learning [32][33][34][35][36] that are well-suited for tackling inverse problems. In particular, we make use of sparse identication of nonlinear dynamical systems 30,[32][33][34]37 to distill the dynamics of AFM cantilever interacting with stiff and compliant samples.…”

Section: Introductionmentioning

confidence: 99%

Quantifying nanoscale forces using machine learning in dynamic atomic force microscopy

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Other recent studies also showed that deep learning techniques can be successfully applied to detect complex-shaped objects in microscopy images. Sotres et al 20 used the YOLOv3 object detection model and a Siamese neural network to determine the locations of DNA molecules in AFM images and identify the same molecule in different images. Okunev et al 21 applied a Cascade Mask-RCNN neural network to detect metal nanoparticles in scanning tunneling microscopy (STM) images.…”

Section: Introductionmentioning

confidence: 99%

AI-based atomic force microscopy image analysis allows to predict electrochemical impedance spectra of defects in tethered bilayer membranes

Raila

Penkauskas

Jankunec

et al. 2022

Sci Rep

View full text Add to dashboard Cite

Atomic force microscopy (AFM) image analysis of supported bilayers, such as tethered bilayer membranes (tBLMs) can reveal the nature of the membrane damage by pore-forming proteins and predict the electrochemical impedance spectroscopy (EIS) response of such objects. However, automated analysis involving pore detection in such images is often non-trivial and can require AI-based object detection techniques. The specific object-detection algorithm we used to determine the defect coordinates in real AFM images was a convolutional neural network (CNN). Defect coordinates allow to predict the EIS response of tBLMs populated by the pore-forming toxins using finite element analysis (FEA) modeling. We tested if the accuracy of the CNN algorithm affected the EIS spectral features sensitive to defect densities and other physical parameters of tBLMs. We found that the EIS spectra can be predicted sufficiently well, however, systematic errors of characteristic spectral points were observed and need to be taken into account. Importantly, the comparison of predicted EIS curves with experimental ones allowed to estimate important physical parameters of tBLMs such as the specific resistance of submembrane reservoir. This reservoir separates phospholipid bilayer from the solid support. We found that the specific resistance of the reservoir amounts to $$10^{4.25 \pm 0.10}$$ 10 4.25 ± 0.10 $$\Omega \cdot cm$$ Ω · c m which is approximately two orders of a magnitude higher compared to the specific resistance of the buffer bathing tBLMs studied in this work. We hypothesize that such effect may be related in part due to decreased concentration of ionic carriers in the submembrane due to decreased relative dielectric permittivity in this region.

show abstract

Enabling autonomous scanning probe microscopy imaging of single molecules with deep learning

Abstract: Scanning Probe Microscopies allow investigating surfaces at the nanoscale, in the real space and with unparalleled signal-to-noise ratio. However, these microscopies are not used as much as it would be...

Cited by 30 publications

References 58 publications

Bayesian Active Learning for Scanning Probe Microscopy: From Gaussian Processes to Hypothesis Learning

Bayesian Active Learning for Scanning Probe Microscopy: From Gaussian Processes to Hypothesis Learning

Quantifying nanoscale forces using machine learning in dynamic atomic force microscopy

AI-based atomic force microscopy image analysis allows to predict electrochemical impedance spectra of defects in tethered bilayer membranes

Contact Info

Product

Resources

About