A critical comparison of topology-based pathway analysis methods

Abstract

One of the aims of high-throughput gene/protein profiling experiments is the identification of biological processes altered between two or more conditions. Pathway analysis is an umbrella term for a multitude of computational approaches used for this purpose. While in the beginning pathway analysis relied on enrichment-based approaches, a newer generation of methods is now available, exploiting pathway topologies in addition to gene/protein expression levels. However, little effort has been invested in their critical assessment with respect to their performance in different experimental setups. Here, we assessed the performance of seven representative methods identifying differentially expressed pathways between two groups of interest based on gene expression data with prior knowledge of pathway topologies: SPIA, PRS, CePa, TAPPA, TopologyGSA, Clipper and DEGraph. We performed a number of controlled experiments that investigated their sensitivity to sample and pathway size, threshold-based filtering of differentially expressed genes, ability to detect target pathways, ability to exploit the topological information and the sensitivity to different pre-processing strategies. We also verified type I error rates and described the influence of overexpression of single genes, gene sets and topological motifs of various sizes on the detection of a pathway as differentially expressed. The results of our experiments demonstrate a wide variability of the tested methods. We provide a set of recommendations for an informed selection of the proper method for a given data analysis task.

A Comparative Study of Topology-based Pathway Enrichment Analysis Methods

“…In this study, we undertook a systematic comparison of nine popular such methods using three data sets from gene expression and metabolomics profiling. Compared to existing reviews [23,24,25,36], our comparison leverages the large sample sizes in the two cancer genomic studies, and, in particular, offers important insights for how the nine competitors perform in metabolomics studies, where the focus is on smaller biochemical pathways. Results in Additional File 1 (Figures S2, S3, S5, S6) suggest similar findings as those observed in Figure 3…”

BNrich: A Bayesian network approach to the pathway enrichment analysis

“…Empirical P-values are attained using the permutation test on cell line datasets A375 (Table1) by generating 1000 simulated datasets in which we randomly assigned samples of the base dataset to two groups. This approach actually gives the observed type I error rate [46,54]. The empirical p-value is calculated by the formula below:…”

The Impact of Pathway Database Choice on Statistical Enrichment Analysis and Predictive Modeling

“…To evaluate PT-based methods, we selected the well-known and highly-cited Signaling Pathway Impact Analysis (SPIA) method (Tarca et al, 2008) for two main reasons: firstly, the guidelines outlined by a comparative study on topology-based methods (Ihnatova et al, 2018) recommend the use of SPIA for datasets with properties similar to TCGA (i.e., possessing two well-defined classes, full expression profiles, many samples and numerous differentially expressed genes). Secondly, SPIA has been reported to have a high specificity whilst preserving dependency on topological information (Ihnatova et al, 2018). Because the R/Bioconductor's SPIA package only contains KEGG pathways, we converted the pathway topologies from the three databases used in this work to a custom format in a similar fashion as graphite (Sales et al, 2018) (Supplementary Information).…”

“…The typical approach to combine pathway information with -omics data is via statistical enrichment analysis, also known as pathway enrichment. The task of navigating through the continuously developing variants of enrichment methods has been undertaken by several recent studies which benchmarked the performance of these techniques (Bayerlová et al, 2015;Ihnatova et al, 2018;Lim et al, 2018) and guide users on the choice for their analyses (Fabris et al, 2019;Reimand et al, 2019). While Bateman et al (2014) examined the impact of choice of different subsets of MSigDB on GSEA analysis, it remains unclear what broader impact an integrative pathway meta-database would have for statistical enrichment analysis.…”