Applying a Deep-Learning-Based Keypoint Detection in Analyzing Surface Nanostructures

Abstract

Scanning tunneling microscopy (STM) imaging has been routinely applied in studying surface nanostructures owing to its capability of acquiring high-resolution molecule-level images of surface nanostructures. However, the image analysis still heavily relies on manual analysis, which is often laborious and lacks uniform criteria. Recently, machine learning has emerged as a powerful tool in material science research for the automatic analysis and processing of image data. In this paper, we propose a method for analyzing molecular STM images using computer vision techniques. We develop a lightweight deep learning framework based on the YOLO algorithm by labeling molecules with its keypoints. Our framework achieves high efficiency while maintaining accuracy, enabling the recognitions of molecules and further statistical analysis. In addition, the usefulness of this model is exemplified by exploring the length of polyphenylene chains fabricated from on-surface synthesis. We foresee that computer vision methods will be frequently used in analyzing image data in the field of surface chemistry.

YOLOv8-PoseBoost: Advancements in Multimodal Robot Pose Keypoint Detection

“…YOLOv7-w6-pose [28]: This is a pose estimation model based on YOLOv7, featuring a smaller model size and faster inference speed, suitable for real-time applications.…”

Automated Generation and Analysis of Molecular Images Using Generative Artificial Intelligence Models

“…Recently, the resurgence and rapid development of machine learning (ML) techniques have opened up unprecedented possibilities for materials science research. − For instance, deep neural networks have shown successes in high-level microscopy vision tasks including image recognition, , semantic segmentation, super-resolution imaging, , image simulations, identifying the atomic structures in atomically resolved AFM images, and denoising STM images of graphene, offering deep insights and providing new discoveries. , …”

Automated Structure Discovery for Scanning Tunneling Microscopy

“…The same problems have been experienced in the field of noncontact AFM where recent advances in machine learning image analysis have been proven effective in structure discovery and chemical identification of single molecules and ice structures. − Beyond machine learning, some of the challenges with respect to chemical identification can be overcome by combining tip-enhanced Raman spectroscopy with STM and AFM to achieve quantitative chemical sensitivity . For STM, advanced image analysis methods have so far not been developed for bond-resolved imaging but instead the focus has been on, e.g., defect detection, molecule keypoint detection, surface characterization, and atom manipulation as well as autonomous experiments . Machine learning methods have also been used for automating the tip conditioning and tip functionalization in STM.…”