Beware of External Validation! - A Comparative Study of Several Validation Techniques used in QSAR Modelling

Majumdar, Subhabrata; Basak, Subhash C.

doi:10.2174/1573409914666180426144304

Cited by 25 publications

(18 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We use a multi‐split cross‐validation method to evaluate our QSAR models. For a small set of chemical compounds, external validation based on a single train‐test split is unstable, and a leave‐one‐out (LOO) approach is preferable for model assessment . Furthermore, since the training set is smaller than the full dataset, it gives a biased estimate of the unknown standard error .…”

Section: Methodsmentioning

confidence: 99%

“…This simply means repeating an external validation method a number of times over different random train‐test splits of the data, and taking the average of any metrics obtained over all such splits. Also known as Monte‐Carlo Cross Validation (MCCV), this validation technique provides stable estimates of evaluation metrics, and provided that the number of splits considered is large, gives unbiased estimate of the generalization error, i. e. q 2 …”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Finding Needles in a Haystack: Determining Key Molecular Descriptors Associated with the Blood‐brain Barrier Entry of Chemical Compounds Using Machine Learning

Majumdar

Basak

Lungu

et al. 2019

Molecular Informatics

Self Cite

View full text Add to dashboard Cite

In this paper we used two sets of calculated molecular descriptors to predict blood-brain barrier (BBB) entry of a collection of 415 chemicals. The set of 579 descriptors were calculated by Schrodinger and TopoCluj software. Polly and Triplet software were used to calculate the second set of 198 descriptors. Following this, modelling and a two-deep, repeated external validation method was used for QSAR formulation. Results show that both sets of descriptors individually and their combination give models of reasonable prediction accuracy. We also uncover the effectiveness of a variable selection approach, by showing that for one of our descriptor sets, the top 5 % predictors in terms of random forest variable importance are able to provide a better performing model than the model with all predictors. The top influential descriptors indicate important aspects of molecular structural features that govern BBB entry of chemicals.Keywords: blood-brain barrier · molecular descriptors · variable selection · machine learning · random forest · two-deep cross validation · quantitative structure-activity relationship (QSAR)[a] S. Majumdar

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Finding Needles in a Haystack: Determining Key Molecular Descriptors Associated with the Blood‐brain Barrier Entry of Chemical Compounds Using Machine Learning

Majumdar

Basak

Lungu

et al. 2019

Molecular Informatics

Self Cite

View full text Add to dashboard Cite

show abstract

“…The predictive capacity of the QSAR model is established by considering internal and external validation procedures (Cherkasov et al, 2014). Internal validation was carried out by considering the data which created the model, while external validation was carried out by using the data which were not used for QSAR model building (Majumdar and Basak, 2018). The fitness of the model is reflected in the value of r 2 .…”

Section: Internal Validationmentioning

confidence: 99%

Integrated Ligand and Structure based approaches towards developing novel Janus Kinase 2 inhibitors for the treatment of myeloproliferative neoplasms

Ambili

Sajitha

2020

Preprint

View full text Add to dashboard Cite

Myeloproliferative neoplasms (MPNs) are a group of diseases affecting hematopoiesis in humans. Types of MPNs include Polycythemia Vera (PV), Essential Thrombocythemia (ET) and myelofibrosis. JAK2 gene mutation at 617th position act as a major causative factor for the onset and progression of MPNs. So, JAK2 inhibitors are widely used for the treatment of MPNs. But, increased incidence of adverse drug reactions associated with JAK2 inhibitors acts as a paramount challenge in the treatment of MPNs. Hence, there exists an urgent need for the identification of novel lead molecules with enhanced potency and bioavailability. We employed ligand and structure-based approaches to identify novel lead molecules which could act as JAK2 inhibitors. The dataset for QSAR modeling (ligand-based approach) comprised of 49 compounds. We have developed a QSAR model, which has got statistical as well as biological significance. Further, all the compounds in the dataset were subjected to molecular docking and bioavailability assessment studies. Derivative compounds with higher potency and bioavailability were identified for the best lead molecule present in the dataset by employing chemical space exploration. Dataset and models are available at https://github.com/giribio/agingdataAbstract FigureGraphical abstract

show abstract

“…To this end, the predictive power of the best equation was verified via leave-one-out cross-validation methods [74] and quantified by Equation (8). This method has been explained previously in several articles, and is well known for its extensive use in QSAR studies [75][76][77][78].…”

Section: Quantitative Structure-activity Relationship (Qsar) and Statmentioning

confidence: 99%

Modeling the Antileukemia Activity of Ellipticine-Related Compounds: QSAR and Molecular Docking Study

et al. 2019

View full text Add to dashboard Cite

The antileukemia cancer activity of organic compounds analogous to ellipticine representes a critical endpoint in the understanding of this dramatic disease. A molecular modeling simulation on a dataset of 23 compounds, all of which comply with Lipinski’s rules and have a structure analogous to ellipticine, was performed using the quantitative structure activity relationship (QSAR) technique, followed by a detailed docking study on three different proteins significantly involved in this disease (PDB IDs: SYK, PI3K and BTK). As a result, a model with only four descriptors (HOMO, softness, AC1RABAMBID, and TS1KFABMID) was found to be robust enough for prediction of the antileukemia activity of the compounds studied in this work, with an R2 of 0.899 and Q2 of 0.730. A favorable interaction between the compounds and their target proteins was found in all cases; in particular, compounds 9 and 22 showed high activity and binding free energy values of around −10 kcal/mol. Theses compounds were evaluated in detail based on their molecular structure, and some modifications are suggested herein to enhance their biological activity. In particular, compounds 22_1, 22_2, 9_1, and 9_2 are indicated as possible new, potent ellipticine derivatives to be synthesized and biologically tested.

show abstract

Beware of External Validation! - A Comparative Study of Several Validation Techniques used in QSAR Modelling

Abstract: Results from external validation are too unstable for the datasets we analyzed. Based on our findings, we recommend using the LOO procedure for validating QSAR predictive models built on high-dimensional small-sample data.

Cited by 25 publications

References 0 publications

Finding Needles in a Haystack: Determining Key Molecular Descriptors Associated with the Blood‐brain Barrier Entry of Chemical Compounds Using Machine Learning

Finding Needles in a Haystack: Determining Key Molecular Descriptors Associated with the Blood‐brain Barrier Entry of Chemical Compounds Using Machine Learning

Integrated Ligand and Structure based approaches towards developing novel Janus Kinase 2 inhibitors for the treatment of myeloproliferative neoplasms

Modeling the Antileukemia Activity of Ellipticine-Related Compounds: QSAR and Molecular Docking Study

Contact Info

Product

Resources

About