MolMiner: You Only Look Once for Chemical Structure Recognition

Abstract

Molecular structures are commonly depicted in 2D printed forms in scientific documents such as journal papers and patents. However, these 2D depictions are not machine readable. Due to a backlog of decades and an increasing amount of printed literatures, there is a high demand for translating printed depictions into machinereadable formats, which is known as Optical Chemical Structure Recognition (OCSR). Most OCSR systems developed over the last three decades use a rule-based approach, which vectorizes the depiction based on the interpretation of vectors and nodes as bonds and atoms. Here, we present a practical software called MolMiner, which is primarily built using deep neural networks originally developed for semantic segmentation and object detection to recognize atom and bond elements from documents. These recognized elements can be easily connected as a molecular graph with a distance-based construction algorithm. MolMiner gave state-of-the-art performance on four benchmark data sets and a self-collected external data set from scientific papers. As MolMiner performed similarly well in real-world OCSR tasks with a user-friendly interface, it is a useful and valuable tool for daily applications. The free download links of Mac and Windows versions are available at https://github.com/iipharma/pharmamind-molminer.

DECIMER.ai: an open platform for automated optical chemical structure identification, segmentation and recognition in scientific publications

“…In recent years, deep-learning-based OCSR tools have been developed 16 – 18 in conjunction with remarkable advancements in computer vision and natural language processing 19 , 20 . While several publications have claimed to have developed tools that are capable of recognising chemical depictions with high accuracy, most of these tools are either proprietary or entirely unavailable 16 , 21 – 23 . Among the few open-source OCSR software solutions 15 , 24 , there is no system that combines chemical structure image segmentation, classification, and translation within a comprehensive workflow.…”

Automated molecular structure segmentation from documents using ChemSAM

“…In evaluating the segmentation capabilities of MolMiner-ImgDet [ 20 ], DECIMER [ 18 ], and ChemSAM, we reference several considerations due to the varied availability of model implementations and the specific challenges presented by chemical structure segmentation. These considerations include the completeness of segmentation, the proportion of structures accurately segmented from the document layout, the recognition rate of colored structures, and the proportion of non-structural elements.…”

DECIMER.ai - An open platform for automated optical chemical structure identification, segmentation and recognition in scientific publications

“…There have been closed-source projects like CLiDE 39 or the recently published MolMiner 23 that combine a segmentation step with an OCSR step in their workflow. CLiDE is a fully commercial tool, MolMiner permits limited access to registered users and offers unlimited access to users who wish to obtain an enterprise licence.…”

“…In recent years, deep-learning-based OCSR tools have been developed 16,17,18 in conjunction with remarkable advancements in computer vision and natural language processing 19,20 . While several publications have claimed to have developed tools that are capable of recognizing chemical depictions with high accuracy, most of these tools are either proprietary or entirely unavailable 16,[21][22][23] . Among the few open-source OCSR software solutions 15,24 , there is no system that combines chemical structure image segmentation, classification, and translation within a comprehensive workflow.…”