Predicting molecular activity on nuclear receptors by multitask neural networks

Valsecchi, Cecile; Collarile, Magda; Grisoni, Francesca; Todeschini, Roberto; Ballabio, Davide; Consonni, Viviana

doi:10.1002/cem.3325

Cited by 19 publications

(17 citation statements)

References 56 publications

(72 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Approaches based on human-engineered molecular descriptors resulted to outperform deep learning based on graphs or SMILES, with no machine learning strategy being consistently better at handling activity cliffs compared to their absolute performance. Our results corroborate previous evidence showing that deep learning methods do not necessarily hold up (yet) against simpler machine learning methods for drug discovery purposes [15][16][17] . We envision the development of deep learning strategies that are (a) more efficient in low-data scenarios and (b) better suited to capture structure-activity "discontinuities" to be key for future prospective applications.…”

Section: Discussionsupporting

confidence: 90%

“…Most AI breakthroughs in chemistry have been driven by deep learning, based on neural networks with multiple processing layers [12][13][14] . However, there is currently no consensus on whether deep learning models outperform simpler machine learning approaches when it comes to molecular property prediction [15][16][17] . The identification of current gaps in machine/deep learning approaches would allow the development of more reliable and widely applicable models to accelerate molecule discovery.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Exposing the limitations of molecular machine learning with activity cliffs.

Tilborg

Alenicheva²,

Grisoni

2022

Preprint

View full text Add to dashboard Cite

Machine learning has become a crucial tool in drug discovery and chemistry at large, e.g., to predict molecular properties, such as bioactivity, with high levels of accuracy. However, activity cliffs – pairs of molecules that are highly similar in their structure but exhibit large differences in potency – have been underinvestigated for their effect on model performance. Not only are these edge cases informative for molecule discovery and optimization, but models that are well-equipped to accurately predict the potency of activity cliffs have an increased potential for prospective applications. Our work aims to fill the current knowledge gap on best-practice machine learning methods in the presence of activity cliffs. We benchmarked more than 20 machine and deep learning approaches on curated bioactivity data from 30 macromolecular targets for their performance on activity cliff compounds. While all methods struggled in the presence of activity cliffs, machine learning approaches based on molecular descriptors outperformed more complex deep learning methods. These results advocate for (a) the inclusion of dedicated “activity-cliff-centered” metrics during model development and evaluation, and (b) the development of novel algorithms to better predict the properties of activity cliff. To this end, the methods, metrics, and results of this study have been encapsulated into an open-access benchmarking platform named MoleculeACE (Activity Cliff Estimation, available on GitHub at: https://github.com/molML/MoleculeACE). MoleculeACE is designed to steer the community towards addressing the pressing but overlooked limitation of molecular machine learning models posed by activity cliffs.

show abstract

Section: Discussionsupporting

confidence: 90%

Section: Introductionmentioning

confidence: 99%

Exposing the limitations of molecular machine learning with activity cliffs.

Tilborg

Alenicheva²,

Grisoni

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Approaches based on human-engineered molecular descriptors resulted to outperform deep learning based on graphs or SMILES, with no machine learning strategy being consistently better at handling activity cliffs compared to their absolute performance. Our results corroborate previous evidence showing that deep learning methods do not necessarily hold up against simpler machine learning methods (yet) for drug discovery purposes [15][16][17] . We envision the development of deep learning strategies that are (a) more efficient in low-data scenarios and (b) better suited to capture structure-activity "discontinuities" to be key for future prospective applications.…”

Section: Discussionmentioning

confidence: 99%

“…Most AI breakthroughs in chemistry have been driven by deep learning -based on neural networks with multiple processing layers [12][13][14] . However, there is currently no consensus on whether deep learning models outperform simpler machine learning approaches when it comes to molecular property prediction [15][16][17] . The identification of current gaps in machine and deep learning approaches would allow the development of more reliable and widely applicable models to accelerate molecule discovery.…”

Section: Introductionmentioning

confidence: 99%

Exposing the limitations of molecular machine learning with activity cliffs.

Tilborg

Alenicheva²,

Grisoni

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Machine learning has become a crucial tool in drug discovery and chemistry at large, e.g., to predict molecular properties, such as bioactivity, with high accuracy. However, activity cliffs – pairs of molecules that are highly similar in their structure but exhibit large differences in potency – have been underinvestigated for their effect on model performance. Not only are these edge cases informative for molecule discovery and optimization, but models that are well-equipped to accurately predict the potency of activity cliffs have an increased potential for prospective applications. Our work aims to fill the current knowledge gap on best-practice machine learning methods in the presence of activity cliffs. We benchmarked a total of 720 machine and deep learning models on curated bioactivity data from 30 macromolecular targets for their performance on activity cliff compounds. While all methods struggled in the presence of activity cliffs, machine learning approaches based on molecular descriptors outperformed more complex deep learning methods. Our findings highlight large case-by-case differences in performance, advocating for (a) the inclusion of dedicated “activity-cliff-centered” metrics during model development and evaluation, and (b) the development of novel algorithms to better predict the properties of activity cliffs. To this end, the methods, metrics, and results of this study have been encapsulated into an open-access benchmarking platform named MoleculeACE (Activity Cliff Estimation, available on GitHub at: https://github.com/molML/MoleculeACE). MoleculeACE is designed to steer the community towards addressing the pressing but overlooked limitation of molecular machine learning models posed by activity cliffs.

show abstract

“…In this work, each bioactivity type for a given receptor was considered as a task. As in [ 22 ], only active and inactive annotations were considered and tasks containing such annotations for less than 200 molecules were discarded. The considered dataset is therefore composed of a total of 14,963 chemicals annotated (as active or inactive) for at least one of the selected 30 tasks.…”

Section: Methodsmentioning

confidence: 99%

Parsimonious Optimization of Multitask Neural Network Hyperparameters

et al. 2021

Self Cite

View full text Add to dashboard Cite

Neural networks are rapidly gaining popularity in chemical modeling and Quantitative Structure–Activity Relationship (QSAR) thanks to their ability to handle multitask problems. However, outcomes of neural networks depend on the tuning of several hyperparameters, whose small variations can often strongly affect their performance. Hence, optimization is a fundamental step in training neural networks although, in many cases, it can be very expensive from a computational point of view. In this study, we compared four of the most widely used approaches for tuning hyperparameters, namely, grid search, random search, tree-structured Parzen estimator, and genetic algorithms on three multitask QSAR datasets. We mainly focused on parsimonious optimization and thus not only on the performance of neural networks, but also the computational time that was taken into account. Furthermore, since the optimization approaches do not directly provide information about the influence of hyperparameters, we applied experimental design strategies to determine their effects on the neural network performance. We found that genetic algorithms, tree-structured Parzen estimator, and random search require on average 0.08% of the hours required by grid search; in addition, tree-structured Parzen estimator and genetic algorithms provide better results than random search.

show abstract

Predicting molecular activity on nuclear receptors by multitask neural networks

Cited by 19 publications

References 56 publications

Exposing the limitations of molecular machine learning with activity cliffs.

Exposing the limitations of molecular machine learning with activity cliffs.

Exposing the limitations of molecular machine learning with activity cliffs.

Parsimonious Optimization of Multitask Neural Network Hyperparameters

Contact Info

Product

Resources

About