Learning curves for drug response prediction in cancer cell lines

Abstract

Background Motivated by the size and availability of cell line drug sensitivity data, researchers have been developing machine learning (ML) models for predicting drug response to advance cancer treatment. As drug sensitivity studies continue generating drug response data, a common question is whether the generalization performance of existing prediction models can be further improved with more training data. Methods We utilize empirical learning curves for evaluating and comparing the data scaling properties of two neural networks (NNs) and two gradient boosting decision tree (GBDT) models trained on four cell line drug screening datasets. The learning curves are accurately fitted to a power law model, providing a framework for assessing the data scaling behavior of these models. Results The curves demonstrate that no single model dominates in terms of prediction performance across all datasets and training sizes, thus suggesting that the actual shape of these curves depends on the unique pair of an ML model and a dataset. The multi-input NN (mNN), in which gene expressions of cancer cells and molecular drug descriptors are input into separate subnetworks, outperforms a single-input NN (sNN), where the cell and drug features are concatenated for the input layer. In contrast, a GBDT with hyperparameter tuning exhibits superior performance as compared with both NNs at the lower range of training set sizes for two of the tested datasets, whereas the mNN consistently performs better at the higher range of training sizes. Moreover, the trajectory of the curves suggests that increasing the sample size is expected to further improve prediction scores of both NNs. These observations demonstrate the benefit of using learning curves to evaluate prediction models, providing a broader perspective on the overall data scaling characteristics. Conclusions A fitted power law learning curve provides a forward-looking metric for analyzing prediction performance and can serve as a co-design tool to guide experimental biologists and computational scientists in the design of future experiments in prospective research studies.

How to Use Machine Learning to Improve the Discrimination between Signal and Background at Particle Colliders

“…The first is a wrapper that includes some of the newest techniques that facilitate the training of NNs, while the second, as mentioned in the introduction, is used for tracking. As a final remark, the benefits NNs can bring over other methods, such as BDTs, not only concern physics, but also appear in fields such as medicine [62,63], finance [64], marketing [65], biology [66] or engineering [67]. As a general trend, NNs tend to perform better in these references, although the present developments in, e.g., BDTs, make these very competitive.…”

A systematic literature review for the prediction of anticancer drug response using various machine‐learning and deep‐learning techniques

“…The year‐wise distribution of the total selected papers can be seen in the graph below in Figure 2, which mainly emphasizes predicting the anticancer drug response (Adam et al, 2020; Ahmadi Moughari & Eslahchi, 2020; Choi et al, 2020; Koch et al, 2020; Kong et al, 2020; Kurilov et al, 2020; Patel et al, 2020; Sharma & Rani, 2020a; Wang, Li, Carpenter, & Guan, 2020; Zhu et al, 2020). The years between 2020 and 2021 (Cilluffo et al, 2021; Feng et al, 2021; Franco et al, 2021; Gerdes et al, 2021; Kim et al, 2021; Lv et al, 2021; Mudali et al, 2020; Nguyen et al, 2021; Partin et al, 2021; Patel & Shah, 2021; Piroozmand et al, 2020; Rafique et al, 2021; Schperberg et al, 2020; Vatansever et al, 2021; Vougas et al, 2020; Wang, Li, & Guan, 2020; Yu et al, 2021; Zhang et al, 2021) recorded the maximum number of publications up to 21–28 articles, while 2013 had no publications in the selected criteria.…”

AI-accelerated protein-ligand docking for SARS-CoV-2 is 100-fold faster with no significant change in detection

“…We performed learning curve analysis with the 3CLPro receptor to determine the training behavior of the model 61 . A subset of 2 M samples were obtained from the full set of 6 M samples.…”