Effects of inductive bias on computational evaluations of ligand-based modeling and on drug discovery

Abstract

Inductive bias is the set of assumptions that a person or procedure makes in making a prediction based on data. Different methods for ligand-based predictive modeling have different inductive biases, with a particularly sharp contrast between 2D and 3D similarity methods. A unique aspect of ligand design is that the data that exist to test methodology have been largely man-made, and that this process of design involves prediction. By analyzing the molecular similarities of known drugs, we show that the inductive bias of the historic drug discovery process has a very strong 2D bias. In studying the performance of ligand-based modeling methods, it is critical to account for this issue in dataset preparation, use of computational controls, and in the interpretation of results. We propose specific strategies to explicitly address the problems posed by inductive bias considerations.

Recommendations for evaluation of computational methods

“…b. All the actives are chemically similar [2,4,5,8,10]. This is more relevant to ligand-based methods, but also applicable to docking because operationally finding chemically similar molecules as being potentially active is of little value in that these will likely be found by other methods.…”

“…Quite frequently, test cases are constructed where both the input ligands and testing ligands are all trivial analogs of a common central structure [2,8,11]. This stems, in part, from the simple fact that the ligands available for constructing tests are most frequently synthesized as part of a design process in which creating analogs is a useful exercise.…”

“…However, construction of such cases can be done, for example, by choosing input ligands that were discovered long before the test ligands or by choosing input ligands that have substantially different overall biological properties (e.g. side effects) than the test ligands [2].…”

“…There is consensus that the degree of ''obvious similarity'' among actives has important effects, particularly in evaluating ligandbased methods [1,2,4,7], but there is less agreement on how to either measure this or to control for it. Our recommendation is that such effects should be quantified in reports, where possible, by, for example, using 2D similarity methods to provide a baseline for the difficulty of a retrieval task or to provide a numerical characterization of the diversity of active ligand sets [2]. In addition, suggestions are made in this issue to either use only single representatives of a chemical class or to weight each according to its order of discovery [1,4,6].…”

“…This special issue of the Journal of Computer-Aided Molecular Design includes eleven papers, each of which makes a detailed study of at least one aspect of methodological evaluation [1][2][3][4][5][6][7][8][9][10][11]. The papers collected within this issue make the detailed case for the recommendations that follow; the recommendations are intended to provide guidance to editorial boards and reviewers of work submitted for publication in our field.…”