Model Choice Metrics to Optimize Profile-QSAR Performance

Abstract

Predicting molecular activity against protein targets is difficult because of the paucity of experimental data. Approaches like multitask modeling and collaborative filtering seek to improve model accuracy by leveraging results from multiple targets, but are limited because different compounds are measured with different assays, leading to sparse data matrices. Profile-QSAR (pQSAR) 2.0 addresses this problem by fitting a series of partial least squares models for each target, using as features the predictions from single-task models on the remaining targets. This method has been shown to produce better results than single task and multitask models. However, the factors determining the success of pQSAR 2.0 have as yet not been characterized. In this paper we examine the experimental conditions that lead to better pQSAR models. We limit the amount of data available to the method by retraining with decreasing amounts of data and explore the model's ability to generalize to compounds that have never been assayed. Finally, we look at the properties of training data needed to demonstrate pQSAR improvement. We apply pQSAR 2.0 on a collection of GPCR and safety targets collected from Drug Target Commons, ExcapeDB, and ChEMBL. We found that pQSAR improved models on 34 of the 149 assays selected. In the other 115 assays, single task random forests offered better performance. There are many factors that contribute to an increase in performance, but the main factor is compound assay coverage. The pQSAR model improves when more compounds are measured in multiple assays. It is necessary to consider the available data before applying pQSAR. Successful pQSAR models require a profile made of correlated targets that share compounds with other assays. This technique is best used when experimental data is available as random forest regressors often do not generalize well enough for virtual drug search applications.