AEcroscopy: A Software–Hardware Framework Empowering Microscopy Toward Automated and Autonomous Experimentation

Abstract

Microscopy has been pivotal in improving the understanding of structure‐function relationships at the nanoscale and is by now ubiquitous in most characterization labs. However, traditional microscopy operations are still limited largely by a human‐centric click‐and‐go paradigm utilizing vendor‐provided software, which limits the scope, utility, efficiency, effectiveness, and at times reproducibility of microscopy experiments. Here, a coupled software–hardware platform is developed that consists of a software package termed AEcroscopy (short for Automated Experiments in Microscopy), along with a field‐programmable‐gate‐array device with LabView‐built customized acquisition scripts, which overcome these limitations and provide the necessary abstractions toward full automation of microscopy platforms. The platform works across multiple vendor devices on scanning probe microscopes and electron microscopes. It enables customized scan trajectories, processing functions that can be triggered locally or remotely on processing servers, user‐defined excitation waveforms, standardization of data models, and completely seamless operation through simple Python commands to enable a plethora of microscopy experiments to be performed in a reproducible, automated manner. This platform can be readily coupled with existing machine‐learning libraries and simulations, to provide automated decision‐making and active theory‐experiment optimization to turn microscopes from characterization tools to instruments capable of autonomous model refinement and physics discovery.

Synergizing human expertise and AI efficiency with language model for microscopy operation and automated experiment design *

“…ChatGPT4 is an advanced model in the LLM landscape [1,4]. AEcroscopy is a custom-developed API, a cross-platform software-hardware system designed for microscope automation [43]. For readers interested in AEcroscopy, we refer to the recent publication [43] where details of AEcroscopy are extensively discussed.…”

“…AEcroscopy is a custom-developed API, a cross-platform software-hardware system designed for microscope automation [43]. For readers interested in AEcroscopy, we refer to the recent publication [43] where details of AEcroscopy are extensively discussed. We feed a complete guideline [44] of AEcroscopy into ChatGPT4 as a prompting resource.…”