Overcoming analytical reliability issues in clinical proteomics using rank-based network approaches

Abstract

(176 words)Proteomics is poised to play critical roles in clinical research. However, due to limited coverage and high noise, integration with powerful analysis algorithms is necessary. In particular, network-based algorithms can improve selection of reproducible features in spite of incomplete proteome coverage, technical inconsistency or high inter-sample variability. We define analytical reliability on three benchmarks ---precision/recall rates, feature-selection stability and crossvalidation accuracy. Using these, we demonstrate the insufficiencies of commonly used Student's ttest and Hypergeometric enrichment. Given advances in sample sizes, quantitation accuracy and coverage, we are now able to introduce and evaluate Ranked-Based Network Approaches (RBNAs) for the first time in proteomics. These include SNET (SubNETwork), FSNET (FuzzySNET), PFSNET (PairedFSNET). We also introduce for the first time, PPFSNET(samplePairedPFSNET), which is a paired-sample variant of PFSNET. RBNAs (particularly PFSNET and PPFSNET) excelled on all three benchmarks and can make consistent and reproducible predictions even in the small-sample size scenario (n=4). Given these qualities, RBNAs represent an important advancement in network biology, and is expected to see practical usage, particularly in clinical biomarker and drug target prediction.

Fuzzy-FishNet: A highly precise distribution-free network approach for feature selection in clinical proteomics

“…In the second concern, as mentioned, while the most powerful RBNAs (PFSNET and PPFSNET) exhibit very high feature-selection stability, as well as precision-recall rates (14), it should be noted they also report a relatively large number of features as well. Significant features selected by PFSNET and PPFSNET have a rather flat p-value distribution i.e., many of the features had p-values at 0 or close to 0 (Supplementary Data 1) thus making it difficult to prioritize which features to test and validate experimentally.…”

“…The existing suite of RBNAs include SubNetworks (SNET), Fuzzy SNet (FSNET), Paired FSNet (PFSNET) and class-Paired PFSNet (PPFSNET). PFSNET and PPFSNET were the two best techniques, and performed very well on all performance benchmarks (14). However, there are two limitations worth investigating further --- 1/ feature selection based on the modified t-statistic and t-distribution may not be a valid assumption and 2/ PFSNET and PPFSNET tend to be extremely sensitive, making a fairly large number of predictions for which the p-value distribution is relatively flat (many of these are 0 or close to 0), this makes it difficult to prioritize which subnets to test and validate first.…”

“…Networks can be combined with proteomics synergistically to overcome its idiosyncratic data issues (5-10). For example, coverage and reliability of predictions can be improved dramatically simply using subnets (short for subnetworks) as contextualization (11-14). In particular, using the recently published clear cell renal cancer SWATH dataset of Guo et al (4), the efficacy of a suite of novel network-based analysis techniques termed Rank-Based Network Approaches (RBNAs) was demonstrated.…”

Integrating Networks and Proteomics: Moving Forward

“…Going further, Boyanova et al showed that network-based categorization of proteomics data identifies network modules that correspond to cell-type origin [14]. Goh et al showed that networks could reproducibly (and robustly) select feature sets with strong class-discrimination power [15] (Box 2). Furthermore, network mining, based on protein complexes, robustly recovered otherwise-undiscovered proteins [9,16].…”

“…Instead of individual proteins, analysis at the level of networks can boost confidence levels (Box 3). Statistical evaluation, not only on significance, but also on reproducibilityfor example incorporating feature-selection stability test [3,15] or irreproducible discovery rate (IDR) [32] should become common practice.…”