ALipSol: An Attention-Driven Mixture-of-Experts Model for Lipophilicity and Solubility Prediction

Abstract

Lipophilicity (logD) and aqueous solubility (logS w) play a central role in drug development. The accurate prediction of these properties remains to be solved due to data scarcity. Current methodologies neglect the intrinsic relationships between physicochemical properties and usually ignore the ionization effects. Here, we propose an attention-driven mixture-of-experts (MoE) model named ALipSol, which explicitly reproduces the hierarchy of task relationships. We adopt the principle of divide-and-conquer by breaking down the complex end point (logD or logS w) into simpler ones (acidic pK a, basic pK a, and logP) and allocating a specific expert network for each subproblem. Subsequently, we implement transfer learning to extract knowledge from related tasks, thus alleviating the dilemma of limited data. Additionally, we substitute the gating network with an attention mechanism to better capture the dynamic task relationships on a per-example basis. We adopt local fine-tuning and consensus prediction to further boost model performance. Extensive evaluation experiments verify the success of the ALipSol model, which achieves RMSE improvement of 8.04%, 2.49%, 8.57%, 12.8%, and 8.60% on the Lipop, ESOL, AqSolDB, external logD, and external logS data sets, respectively, compared with Attentive FP and the state-of-the-art in silico tools. In particular, our model yields more significant advantages (Welch’s t-test) for small training data, implying its high robustness and generalizability. The interpretability analysis proves that the atom contributions learned by ALipSol are more reasonable compared with the vanilla Attentive FP, and the substitution effects in benzene derivatives agreed well with empirical constants, revealing the potential of our model to extract useful patterns from data and provide guidance for lead optimization.

Comment on ‘Physics-based representations for machine learning properties of chemical reactions’

“…For example, sometimes classical ML techniques as simple as decision trees or random forest can be quite effective at estimating kinetic parameters [28] or ranking reaction conformations [29], and we do not doubt that there are some molecular problems and/or datasets where KRR or other architectures may have some advantages over GNN-based approaches. Our argument instead is that comparing model performance to relevant baselines-and to experimental data when available [30][31][32][33][34][35][36][37][38][39][40][41][42]-should never be 'beyond the scope' of work for a model-developer. Rather, it is an essential first step at convincing readers that the model is useful.…”

The blobulator: a webtool for identification and visual exploration of hydrophobic modularity in protein sequences

“…Most tools that extract information innate to protein sequences either utilize the structural hierarchy framework or work without a hierarchical framework. Examples include predicting disease-associated mutations [6][7][8][9][10][11][12], predicting solubility [13][14][15], detecting aggregating regions [16][17][18][19][20], predicting intrinsic disorder [18,[21][22][23][24][25], designing protein sequences with specific features [26][27][28], and comparing sequences [29,30]. Some of these tools use a fixed-width moving window to define local sequence context, which artificially places the considered residue at the center of its "local sequence" and ignores any natural boundaries present within proteins.…”

Enhancing Molecular Property Prediction through Task-Oriented Transfer Learning: Integrating Universal Structural Insights and Domain-Specific Knowledge

“…This entails identifying promising compounds from a large pool of molecules and receiving ADMET feedback before actual synthesis. To address these issues, the development of quantitative structure–property relationship (QSPR) models, using computer technology to predict ADMET properties, has emerged as a cost-effective and efficient alternative. − …”