DECIMER.ai: an open platform for automated optical chemical structure identification, segmentation and recognition in scientific publications

Abstract

The number of publications describing chemical structures has increased steadily over the last decades. However, the majority of published chemical information is currently not available in machine-readable form in public databases. It remains a challenge to automate the process of information extraction in a way that requires less manual intervention - especially the mining of chemical structure depictions. As an open-source platform that leverages recent advancements in deep learning, computer vision, and natural language processing, DECIMER.ai (Deep lEarning for Chemical IMagE Recognition) strives to automatically segment, classify, and translate chemical structure depictions from the printed literature. The segmentation and classification tools are the only openly available packages of their kind, and the optical chemical structure recognition (OCSR) core application yields outstanding performance on all benchmark datasets. The source code, the trained models and the datasets developed in this work have been published under permissive licences. An instance of the DECIMER web application is available at https://decimer.ai.

Comparing software tools for optical chemical structure recognition

“…2 Other publications have used the same exact prediction and the Tanimoto similarity as an additional metric. 4,7 SwinOCSR also uses BLEU and ROUGE 7 which are N-gram based precision methods developed for machine translation. OCMR has used the Levenshtein distance between the predicted SMILES string and the ground truth SMILES to quantify the dissimilarity of the predicted SMILES.…”

“…In contrast, the conversion of images to chemical structures (optical chemical structure recognition, OCSR) still represents a signicant challenge for established soware tools like Kekulé, CLiDE, OSRA and others described by Rajan et al 1 In the recent past, the development of articial intelligence (AI) such as transformer based machine learning tools has ignited a novel interest in image processing and the creation of generative models that lead to the rapid development of novel applications for predicting chemical structures from their images. Such AI based image-to-structure methods are Mol-Scribe 2 and RxnScribe, 3 DECIMER, 4 ReactionDataExtractor, 5 Img2Mol, 6 SwinOCSR 7 and OCMR 8 that have recently become available and that were shown to outperform previous rule based, analytical methods both in recall as well as in precision.…”

pKalculator: A pKa predictor for C-H bonds

“…As the iBonD database only contains an image of each molecule, we employ the tool Deep Learning for Chemical Image Recognition software (DECIMER v. 2.0), developed by Rajan et al. [8][9][10] While DECIMER converts molecular images into SMILES, manual intervention is required to ensure the SMILES string correctly represents the molecule. Finally, to mirror the dataset by Roszak et al, 3 we also incorporate 43 heterocycles without experimental pK a values from Shen et al, leaving us with a dataset of 775 compounds.…”

Image2InChI: Automated Molecular Optical Image Recognition

“…In 2023, DECIMER.ai, an open-source platform, aimed at addressing the challenge of identifying, segmenting, and recognizing chemical structure depictions in scientific literature . The platform incorporates DECIMER Segmentation, a toolkit that utilizes Mask R-CNN25 for detecting and segmenting chemical structures.…”