A comprehensive performance evaluation on the prediction results of existing cooperative transcription factors identification algorithms

Abstract

BackgroundEukaryotic transcriptional regulation is known to be highly connected through the networks of cooperative transcription factors (TFs). Measuring the cooperativity of TFs is helpful for understanding the biological relevance of these TFs in regulating genes. The recent advances in computational techniques led to various predictions of cooperative TF pairs in yeast. As each algorithm integrated different data resources and was developed based on different rationales, it possessed its own merit and claimed outperforming others. However, the claim was prone to subjectivity because each algorithm compared with only a few other algorithms and only used a small set of performance indices for comparison. This motivated us to propose a series of indices to objectively evaluate the prediction performance of existing algorithms. And based on the proposed performance indices, we conducted a comprehensive performance evaluation.ResultsWe collected 14 sets of predicted cooperative TF pairs (PCTFPs) in yeast from 14 existing algorithms in the literature. Using the eight performance indices we adopted/proposed, the cooperativity of each PCTFP was measured and a ranking score according to the mean cooperativity of the set was given to each set of PCTFPs under evaluation for each performance index. It was seen that the ranking scores of a set of PCTFPs vary with different performance indices, implying that an algorithm used in predicting cooperative TF pairs is of strength somewhere but may be of weakness elsewhere. We finally made a comprehensive ranking for these 14 sets. The results showed that Wang J's study obtained the best performance evaluation on the prediction of cooperative TF pairs in yeast.ConclusionsIn this study, we adopted/proposed eight performance indices to make a comprehensive performance evaluation on the prediction results of 14 existing cooperative TFs identification algorithms. Most importantly, these proposed indices can be easily applied to measure the performance of new algorithms developed in the future, thus expedite progress in this research field.

Identifying cooperative transcription factors in yeast using multiple data sources

“…Following previous studies in the literature [ 3 , 4 , 8 , 24 ], we evaluated cooperativity between two TFs in a PCTFP based on the rationale: the similarity of protein-protein interactions (PPI) partners between two TFs suggests that they contribute to the same biological processes and participate in the same regulatory mechanism. The physical PPI data were downloaded from the BioGRID database [ 25 ].…”

MVIAeval: a web tool for comprehensively evaluating the performance of a new missing value imputation algorithm

“…In MVIAeval, we implemented two existing comprehensive performance scores [48, 49] to provide the overall performance comparison results for the selected benchmark microarray datasets and performance indices. The first one, termed the overall ranking score (ORS), is defined as the sum of the rankings of an algorithm for the selected performance indices and benchmark microarray datasets [48, 49].…”

“…The first one, termed the overall ranking score (ORS), is defined as the sum of the rankings of an algorithm for the selected performance indices and benchmark microarray datasets [48, 49]. The ranking of an algorithm for a specific performance index and a specific benchmark microarray dataset is d if its performance ranks #d among all the compared algorithms.…”

“…The other comprehensive performance score, termed the overall normalized score (ONS), is calculated by the sum of the normalized scores for the benchmark microarray datasets and performance indices [48, 49]. The ONS of the algorithm k is calculated like the following:where S ij ( k ) and N ij ( k ) is the original score and the normalized score of the algorithm k for the selected performance index i and benchmark microarray dataset j , respectively; I is the number of the selected indices; J is the number of the selected benchmark microarray datasets and m is the number of the algorithms being compared.…”

InCoB2014: Systems Biology update from the Asia-Pacific

“…A comprehensive understanding of the symbiosis between the human microbiome and the host organism is essential for defining its role in human health and disease. Yang et al [ 13 ] have applied an ensemble clustering framework to delineate the structure of human microbiome and provide a new insight to the pathological role of microbes within the host organism. Srihari et al [ 14 ] have analysed complexes in core cellular processes to decipher cancer mechanisms, by data integration at the protein-protein interaction and gene expression levels, across all cancer conditions.…”