Comment on ‘Physics-based representations for machine learning properties of chemical reactions’

Abstract

In a recent article in this journal, van Gerwen et al (2022 Mach. Learn.: Sci. Technol. 3 045005) presented a kernel ridge regression model to predict reaction barrier heights. Here, we comment on the utility of that model and present references and results that contradict several statements made in that article. Our primary interest is to offer a broader perspective by presenting three aspects that are essential for researchers to consider when creating models for chemical kinetics: (1) are the model’s prediction targets and associated errors sufficient for practical applications? (2) Does the model prioritize user-friendly inputs so it is practical for others to integrate into prediction workflows? (3) Does the analysis report performance on both interpolative and more challenging extrapolative data splits so users have a realistic idea of the likely errors in the model’s predictions?

Graph to Activation Energy Models Easily Reach Irreducible Errors but Show Limited Transferability

“…Graph-based neural-network models have become notorious in many contexts for overfitting and poor out-of-distribution performance. ,, Although the models trained on RGD1 show excellent testing performance on unseen reactions, this is a large data set, and reactions typically involve a small number of bond changes and conserved mechanisms. This means that even if the testing set involves unseen reactions in terms of reactants or products, it is not expected to necessarily present novel reactivity (e.g., in terms of new types of bonds being broken and formed) that is not seen elsewhere in the training data.…”

“…The negotiation of these tradeoffs remains a live issue. 44,56,57 Despite many practical demonstrations of the graph-toactivation-energy (G2Ea) concept, several challenges persist that limit the usefulness of these models as drop-in replacements for quantum-chemistry-based TS searches. One challenge is that the scarcity of large reaction data sets has limited convincing out-of-distribution tests of the transferability of G2Ea models.…”

“…In this sense, the ideal model would be able to achieve high accuracy based solely on the reactant and product graphs. The negotiation of these trade-offs remains a live issue. ,, …”

Subgraph Isomorphic Decision Tree to Predict Radical Thermochemistry with Bounded Uncertainty Estimation

“…The random splitting approach employed above assesses the interpolation ability of the tree models. 55 Here, we assess the extrapolation ability of the empirical tree model and the SIDT model with a more challenging data split, i.e., cluster split. The details on how we perform the cluster split can be found in Section 3.3.3.…”

CoeffNet: predicting activation barriers through a chemically-interpretable, equivariant and physically constrained graph neural network

“…We nd that the MAE is 0.42 eV on a dataset which spans z10 eV, similar to the MAE of obtained with the SLATM (2) d kernel and nearly double that of a trained directed message-passing neural network for slightly different dataset splits. 40 We do not perform extensive hyperparameter tests, instead we use the same parameters as listed in Section VI; while out of the scope of our current work, we expect that further hyperparameter and model tuning to be benecial to reduce the mean absolute error for this dataset. Furthermore, we nd that the standard deviation of the error to be 0.68 eV (corresponding to a root-mean-square error of 0.65 eV), well within the margin of error expected to accurately predict reactions of interest which can then undergo computationally intensive DFT and transition state nding calculations.…”