Learning Chemistry: Exploring the Suitability of Machine Learning for the Task of Structure-based Chemical Ontology Classification

Abstract

Chemical data is increasingly openly available in databases such as PubChem, which contains more than 110 million compound entries as of October 2020. With the availability of data at such scale, the burden has shifted to organisation, analysis and interpretation. Chemical ontologies provide structured classifications of chemical entities that can be used for navigation and filtering of the large chemical space. ChEBI is a prominent example of a chemical ontology, widely used in life science contexts. However, ChEBI is manually maintained and as such cannot easily scale to the full scope of public chemical data. There is a need for tools that are able to automatically classify chemical data into chemical ontologies, which can be framed as a hierarchical multi-class classification problem. In this paper we evaluate machine learning approaches for this task, comparing different learning frameworks including logistic regression, decision trees and long short-term memory articial neural networks, and different encoding approaches for the chemical structures, including cheminformatics fingerprints and character-based encoding from chemical line notation representations. We nd that classical learning approaches such as logistic regression perform well with sets of relatively specific, disjoint chemical classes, while the neural network is able to handle larger sets of overlapping classes but needs more examples per class to learn from, and is not able to make a class prediction for every molecule. Future work will explore hybrid and ensemble approaches, as well as alternative network architectures including neuro-symbolic approaches.

Automated and Explainable Ontology Extension Based on Deep Learning: A Case Study in the Chemical Domain

“…Table 2 and Fig. 5 compare the results of the current model with the previously obtained results for the LSTM model from [30]. We saw an improvement in performance both when we look at the distributions of values for the molecule-wise F1 scores (Figure 5a) and for the class-wise F1 scores (Figure 5b).…”

“…In our previous work [30], we have evaluated several classifiers for this task, including a long short-term memory (LSTM) model which was the best-performing overall. The results of this effort were satisfactory as a whole, but several specific limitations were identified.…”

“…It is therefore necessary to define a sampling strategy to select leaf node entities and classes to minimize the impact on the training. In order to be able to compare our results to our earlier findings, we have used the same dataset 2 and sampling strategy as was used in Hastings et al [30]. Using only the hierarchical sub-class relations in the ChEBI ontology, this dataset was created by randomly sampling leaf node molecular entities from higher-level classes that they are subclasses of, using an algorithm that aimed to minimize (as far as possible) class overlap, described in Section 3 of [30].…”

“…Transformer-based models rely on a subword tokenization algorithm that counts the occurrences of each character pair in the dataset and incrementally adds the most frequently occurring pairs to the vocabulary. In our previous work, Hastings et al [30], we used two strategies to encode the input sequences for the LSTM model: a character-level tokenization and an atom-wise tokenization, where letter combinations that represent an atom were encoded as a token. In the current work, we use the Byte Pair Encoding (BPE) algorithm as a sub-word tokenization method with a RoBERTa architecture.…”