Siamese Recurrent Neural Network with a Self-Attention Mechanism for Bioactivity Prediction

Fernández-Llaneza, Daniel; Ulander, Silas; Gogishvili, Dea; Nittinger, Eva; Zhao, Hongtao; Tyrchan, Christian

doi:10.1021/acsomega.1c01266

Cited by 21 publications

(30 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We propose twin-network training of deep-learning models as a potential strategy to increase AC-sensitivity. Comparatively little work has been done to investigate twin neural network architectures (also referred to as Siamese networks [9,12,41,70]) in computational drug discovery [3,6,11,18,22,24,36,53,58,61,73,82]. However, twin networks provide a natural way to tackle chemical prediction problems on compound pairs such as AC-classification.…”

Section: Future Research: Exploring Twin-network Training Schemesmentioning

confidence: 99%

Exploring QSAR Models for Activity-Cliff Prediction

Dablander¹,

Hanser²,

Lambiotte³

et al. 2023

Preprint

View full text Add to dashboard Cite

Introduction and Methodology: Pairs of similar compounds that only differ by a small structural modification but exhibit a large difference in their binding affinity for a given target are known as activity cliffs (ACs). It has been hypothesised that QSAR models struggle to predict ACs and that ACs thus form a major source of prediction error. However, a study to explore the AC-prediction power of modern QSAR methods and its relationship to general QSAR-prediction performance is lacking. We systematically construct nine distinct QSAR models by combining three molecular representation methods (extended-connectivity fingerprints, physicochemical-descriptor vectors and graph isomorphism networks) with three regression techniques (random forests, k-nearest neighbours and multilayer perceptrons); we then use each resulting model to classify pairs of similar compounds as ACs or non-ACs and to predict the activities of individual molecules in three case studies: dopamine receptor D2, factor Xa, and SARS-CoV-2 main protease. Results and Conclusions:We observe low AC-sensitivity amongst the tested models when the activities of both compounds are unknown, but a substantial increase in AC-sensitivity when the actual activity of one of the compounds is given. Graph isomorphism features are found to be competitive with or superior to classical molecular representations for AC-classification and can thus be employed as baseline AC-prediction models or simple compound-optimisation tools. For general QSAR-prediction, however, extended-connectivity fingerprints still consistently deliver the best performance. Our results provide strong support for the hypothesis that indeed QSAR methods frequently fail to predict ACs. We propose twin-network training for deep learning models as a potential future pathway to increase AC-sensitivity and thus overall QSAR performance.

show abstract

Section: Future Research: Exploring Twin-network Training Schemesmentioning

confidence: 99%

Exploring QSAR Models for Activity-Cliff Prediction

Dablander¹,

Hanser²,

Lambiotte³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The experimental results show that machine learning plays an important role in screening and reducing HAD prescription errors and has potential benefits. D FernándezLlaneza, S Ulander, D Gogishvili, et al (14) proposed a Siamese recurrent neural network model (SiameseCHEM) based on bidirectional longterm and short-term memory structure with self attention mechanism, which can automatically learn the discriminant features from the SMILES representation of small molecules. Then it is trained with random SMILES strings, which proves that it is robust to binary or classification tasks of biological activity.…”

Section: Related Workmentioning

confidence: 99%

Application of random forest based on semi-automatic parameter adjustment for optimization of anti-breast cancer drugs

Zhang

Kong

et al. 2022

Front. Oncol.

View full text Add to dashboard Cite

The optimization of drug properties in the process of cancer drug development is very important to save research and development time and cost. In order to make the anti-breast cancer drug candidates with good biological activity, this paper collected 1974 compounds, firstly, the top 20 molecular descriptors that have the most influence on biological activity were screened by using XGBoost-based data feature selection; secondly, on this basis, take pIC50 values as feature data and use a variety of machine learning algorithms to compare, soas to select a most suitable algorithm to predict the IC50 and pIC50 values. It is preliminarily found that the effects of Random Forest, XGBoost and Gradient-enhanced algorithms are good and have little difference, and the Support vector machine is the worst. Then, using the Semi-automatic parameter adjustment method to adjust the parameters of Random Forest, XGBoost and Gradient-enhanced algorithms to find the optimal parameters. It is found that the Random Forest algorithm has high accuracy and excellent anti over fitting, and the algorithm is stable. Its prediction accuracy is 0.745. Finally, the accuracy of the results is verified by training the model with the preliminarily selected data, which provides an innovative solution for the optimization of the properties of anti- breast cancer drugs, and can provide better support for the early research and development of anti-breast cancer drugs.

show abstract

“…The generation of the steam temperature has a significant delay characteristic. The long short-term memory network (LSTM) has good performance in time-series forecasting, which solves the problems of gradient disappearance, gradient explosion, and a long sequence dependence in the long sequence training process. , Gupta et al used a single layer of LSTM with 32 nodes to predict fouling in air preheaters, which can be predicted 3 months in advance. Tan et al analyzed the effect of different delay time sequences on the model.…”

Section: Introductionmentioning

confidence: 99%

Dense Residual LSTM-Attention Network for Boiler Steam Temperature Prediction with Uncertainty Analysis

et al. 2022

View full text Add to dashboard Cite

Flexible operation of large-scale boilers for electricity generation is essential in modern power systems. An accurate prediction of boiler steam temperature is of great importance to the operational efficiency of boiler units to prevent the occurrence of overtemperature. In this study, a dense, residual long short-term memory network (LSTM)-attention model is proposed for steam temperature prediction. In particular, the residual elements in the proposed model have a great advantage in improving the accuracy by adding short skip connections between layers. To provide overall information for the steam temperature prediction, uncertainty analysis based on the proposed model is performed to quantify the uncertainties in steam temperature variations. Our results demonstrate that the proposed method exhibits great performance in steam temperature prediction with a mean absolute error (MAE) of less than 0.6 °C. Compared to algorithms such as support-vector regression (SVR), ridge regression (RIDGE), the recurrent neural network (RNN), the gated recurrent unit (GRU), and LSTM, the prediction accuracy of the proposed model outperforms by 32, 16, 12, 10, and 11% in terms of MAE, respectively. According to our analysis, the dense residual LSTM-attention model is shown to provide an accurate early warning of overtemperature, enabling the development of real-time steam temperature control.

show abstract

Siamese Recurrent Neural Network with a Self-Attention Mechanism for Bioactivity Prediction

Cited by 21 publications

References 35 publications

Exploring QSAR Models for Activity-Cliff Prediction

Exploring QSAR Models for Activity-Cliff Prediction

Application of random forest based on semi-automatic parameter adjustment for optimization of anti-breast cancer drugs

Dense Residual LSTM-Attention Network for Boiler Steam Temperature Prediction with Uncertainty Analysis

Contact Info

Product

Resources

About