Autonomous robotic nanofabrication with reinforcement learning

Abstract

The ability to handle single molecules as effectively as macroscopic building blocks would enable the construction of complex supramolecular structures inaccessible to self-assembly. The fundamental challenges obstructing this goal are the uncontrolled variability and poor observability of atomic-scale conformations. Here, we present a strategy to work around both obstacles and demonstrate autonomous robotic nanofabrication by manipulating single molecules. Our approach uses reinforcement learning (RL), which finds solution strategies even in the face of large uncertainty and sparse feedback. We demonstrate the potential of our RL approach by removing molecules autonomously with a scanning probe microscope from a supramolecular structure. Our RL agent reaches an excellent performance, enabling us to automate a task that previously had to be performed by a human. We anticipate that our work opens the way toward autonomous agents for the robotic construction of functional supramolecular structures with speed, precision, and perseverance beyond our current capabilities.

Combining Machine Learning and Computational Chemistry for Predictive Insights Into Chemical Systems

“…More and more research efforts start to combine data-driven learning algorithms with rigorous scientific or engineering theory to yield novel insights and applications. 9 , 15 , 351 …”

“… 383 For chemistry applications, RL techniques are being increasingly used for finding molecules with desired properties in large chemical spaces. 9 …”

Enabling autonomous scanning probe microscopy imaging of single molecules with deep learning

“…Recent studies have already shown the potential of RL in SPM automation. 24,25 The availability of error signals in SPM e.g., the deviation of the signal on which feedback is performed with respect to the operation set point, should further facilitate their implementation. For instance, following an approach where the actions would be the variation of the scanning parameters and the reward the minimization of the error signal.…”

Electrostatic Discovery Atomic Force Microscopy

“…In general, the adoption of ML methods into materials analysis has seen rapid recent growth [28][29][30] and this has been followed by an equivalent growth in its applications to image analysis in SPM [31][32][33][34][35][36]. Here we build upon our ML method for predicting molecular structure from AFM images [37], to predict the electrostatic field of the sample molecule.…”