Pitfalls of machine learning models for protein-protein interactions

Abstract

Protein-protein interactions (PPIs) are essential to understanding biological pathways as well as their roles in development and disease. Computational tools have been successful at predicting PPIs in silico, but the lack of consistent and reliable frameworks for this task has led to network models that are difficult to compare and, overall, a low level of trust in the PPI predictions. To better understand the underlying mechanisms that underpin these models, we designed B4PPI, an open-source framework for benchmarking that accounts for a range of biological and statistical pitfalls while facilitating reproducibility. We use B4PPI to shed light on the impact of network topology and how different algorithms deal with highly connected proteins. By studying functional genomics-based and sequence-based models (the two most popular approaches) on human PPIs, we show their complementarity as the former performs best on lone proteins while the latter specialises in interactions involving hubs. We also show that algorithm design has little impact on performance with functional genomic data. We replicate our results between both human and S. Cerevisiae data and demonstrate that models using functional genomics are better suited to PPI prediction across species. With rapidly increasing amounts of sequence and functional genomics data, our study provides a systematic foundation for future construction, comparison and application of PPI networks.

GREENER principles for environmentally sustainable computational science

“…Some authors already do this voluntarily (for example, refs. [54][55][56][57][58][59], mostly in bioinformatics and machine learning so far, but there is potential to expand it to other areas of computational science. In some instances, showing that a new tool is greener can be an argument in support of a new method 60 .…”

Ten recommendations for reducing the carbon footprint of research computing in human neuroimaging

“…This approach has already been taken by researchers in compute-heavy fields (e.g., Lannelongue & Inouye, 2023b;Xu et al, 2023), and a comprehensive framework for reporting these figures is provided by the Scientific CO 2 nduct initiative ( Sweke et al, 2022; https://scientific -conduct . github .…”

Environmental Impacts of Machine Learning Applications in Protein Science

“…While earlier tools tended to rely on low-power machine learning algorithms, most methods now leverage deep learning and involve longer training times. A recent comparison of machine learning (randomforest)and deep learning(recurrentneu-ral networks) found that in some situations, the deep learning approach could emit 22,000 more GHGs for similar performance (Lannelongue and Inouye 2022). However, runtimes remain small and, in this case, training the deep learning model once had a carbon footprint of 356 gCO 2 e and ∼36 kgCO 2 e (75 kWh) when including fine-tuning the network.…”