Influence of Data Similarity on the Scoring Power of Machine-learning Scoring Functions for Docking

Machine-learning scoring functions trained on complexes dissimilar to the test set already outperform classical counterparts on a blind benchmark

“…Despite the consistency, we advocate the more robust approach by MM-align, first introduced by Sze et al. [ 8 ].…”

“…The approach by Sze et al. [ 8 ] was employed to calculate the similarity in terms of protein structure, ligand fingerprint and pocket topology.…”

“…At a specific cutoff, a complex is excluded from the original full training set if its similarity to any of the test complexes is higher than the cutoff. In other words, a complex is included in the training set if its similarity to every test complex is always no greater than the cutoff [ 8 ]. Mathematically, for both protein structure and ligand fingerprint similarities whose values are normalized to [0, 1], a series of new training sets (NTs) were created by gradually removing complexes from the OT according to varying cut-off values given a fixed test set (TS): \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}\begin{equation*} {NT}_{ds}^s(c)=\left\{\ {p}_i\ |\kern0.50em {p}_i\in OT\ and\ \forall{q}_j\in TS,s\left({p}_i,{q}_j\right)\le c\ \right\} \end{equation*}\end{document} where c is the cutoff; \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}${p}_i$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}${q}_j$\end{document} represent the i th and j th complexes from OT and TS, respectively; and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$s({p}_i,{q}_j)$\end{document} is the similarity between \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}${p}_i$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}${q}_j$\end{document} .…”

“…proposed a revised definition of structural similarity between a pair of training and test set proteins, introduced a different measure of binding pocket similarity, and benchmarked three classical SFs and four RF-based SFs on CASF-2013. They found that even if the training set was split into two halves and the half with proteins dissimilar to the test set was used for training, RF-based SFs still produced a smaller prediction error than the best classical SF, thus confirming that dissimilar training complexes may be valuable when allied with appropriate ML approaches and informative descriptors [ 8 ].…”

“…Here we have expanded the above six studies from the following perspectives. Firstly, we will demonstrate three examples to show that the method employed by four early works [ 3–6 ] for calculating structural similarity could be error prone, hence a revised method proposed lately [ 8 ] should be advocated. Secondly, in addition to CASF-2016, a blind evaluation was conducted too, where only data available until 2017 were used to construct the SFs that predict the binding affinities of complexes released by 2018 as if these had not been measured hitherto [ 1 ].…”