Genome-Scale Computational Approaches to Memory-Intensive Applications in Systems Biology

Zhang, Yun; Abu-Khzam, Faisal N.; Baldwin, Nicole; Chesler, Elissa J.; Langston, Michael A.; Samatova, Nagiza F.

doi:10.1109/sc.2005.29

Cited by 59 publications

(68 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Early works in the area of parallel MCE include Zhang et al [45] and Du et al [12]. Zhang et al developed an algorithm based on the Kose et al [24] algorithm.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Mining maximal cliques from a large graph using MapReduce: Tackling highly uneven subproblem sizes

Svendsen

Mukherjee

Tirthapura

2015

Journal of Parallel and Distributed Computing

View full text Add to dashboard Cite

We consider Maximal Clique Enumeration (MCE) from a large graph. A maximal clique is perhaps the most fundamental dense substructure in a graph, and MCE is an important tool to discover densely connected subgraphs, with numerous applications to data mining on web graphs, social networks, and biological networks. While effective sequential methods for MCE are known, scalable parallel methods for MCE are still lacking.We present a new parallel algorithm for MCE, Parallel Enumeration of Cliques using Ordering (PECO" role="presentation" style="box-sizing: border-box; display: inline-block; line-height: normal; font-size: 14.4px; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; maxwidth: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; padding: 0px; margin: 0px; position: relative;">PECO), designed for the MapReduce framework. Unlike previous works, which required a post-processing step to remove duplicate and non-maximal cliques, PECO" role="presentation" style="boxsizing: border-box; display: inline-block; line-height: normal; font-size: 14.4px; word-spacing: normal; wordwrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; minwidth: 0px; min-height: 0px; border: 0px; padding: 0px; margin: 0px; position: relative;">PECOenumerates only maximal cliques with no duplicates. The key technical ingredient is a total ordering of the vertices of the graph which is used in a novel way to achieve a load balanced distribution of work, and to eliminate redundant work among processors. We implemented PECO" role="presentation" style="box-sizing: border-box; display: inline-block; line-height: normal; font-size: 14.4px; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; padding: 0px; margin: 0px; position: relative;">PECO on Hadoop MapReduce, and our experiments on a cluster show that the algorithm can effectively process a variety of large real-world graphs with millions of vertices and tens of millions of maximal cliques, and scales well with the degree of available parallelism. KeywordsGraph mining, Maximal clique enumeration, Enumeration algorithm, MapReduce, Hadoop, Parallel algorithm, Clique, Load balancing Disciplines Electrical and Computer EngineeringComments This is a manuscript of an article from Svendsen, Michael, Arko Provo Mukherjee, and Srikanta Tirthapura. "Mining maximal cliques from a large graph using mapreduce: Tackling highly uneven subproblem sizes. h i g h l i g h t s• Scalable method for enumerating maximal cliques in a graph using MapReduce.• Effective solution to load balancing.• Experimental evaluation of our solution on large real world graphs.• Outperforms previous MapReduce solutions by orders of magnitude. a r t i c l e i n f o b s t r a c tWe consider Maximal Clique Enumeration (MCE) from a large graph. A maximal clique is perhaps the most fundamental dense substru...

show abstract

“…Early works in the area of parallel MCE include Zhang et al [45] and Du et al [12]. Zhang et al developed an algorithm based on the Kose et al [24] algorithm.…”

Section: Related Workmentioning

confidence: 99%

“…While our algorithm maybe more broadly applicable, in this work we focus our implementation on the widely used MapReduce [10,11,17] framework for cluster computing. While MCE is widely studied in the sequential setting [4,5,8,25,13,23,21,31,40,41], there is relatively less work on parallel methods [45,12,38,43,30].…”

Section: Introductionmentioning

confidence: 99%

Mining maximal cliques from a large graph using MapReduce: Tackling highly uneven subproblem sizes

Svendsen

Mukherjee

Tirthapura

2015

Journal of Parallel and Distributed Computing

View full text Add to dashboard Cite

show abstract

“…A variety of emerging applications are focused on computations over data modeled as a graph: examples include finding groups of actors or communities in social networks [18,22], Web mining [19], entity resolution [26], graph mining [37,41], and detection of consistently co-expressed gene groups in systems biology [27]. For the problems just cited, as well as a number of others, a critical component of the analysis is the detection of cliques (fully connected components) in the structure of the network graph.…”

Section: Introductionmentioning

confidence: 99%

Efficient Maximal Clique Enumeration Over Graph Data

et al. 2016

View full text Add to dashboard Cite

In a wide variety of emerging data-intensive applications, such as social network analysis, Web document clustering, entity resolution, and detection of consistently coexpressed genes in systems biology, the detection of dense subgraphs (cliques) is an essential component. Unfortunately, this problem is NP-Complete and thus computationally intensive at scale-hence there is a need for efficient processing, as well as the techniques for distributing the computation across multiple machines such that the computation, which is too time-consuming on a single machine, can be efficiently performed on a machine cluster given that it is large enough. In this paper, we propose a new algorithm (called GP) for maximal clique enumeration. It identifies cliques by the operation of binary graph partitioning, which iteratively divides a graph until each task is sufficiently small to be processed in parallel. Given a connected graph G ¼ ðV; EÞ, the GP algorithm has a space complexity of O(|E|) and a time complexity of OðjEjlðGÞÞ, where lðGÞ represents the number of different cliques existing in G. We also present a hybrid algorithm, which can effectively leverage the advantages of both the GP algorithm and the classical Bron-and-Kerbosch (BK) algorithm. Then, we develop corresponding parallel solutions based on the GP and hybrid algorithms. Finally, we evaluate the performance of the proposed solutions on real and synthetic graph data. Our extensive experiments show that in both centralized and parallel setting, our proposed GP and hybrid approaches achieve considerably better performance than the state-ofthe-art BK approach. Our parallel solutions are implemented and evaluated on MapReduce, a popular shared-nothing parallel framework, but can easily generalize to other sharednothing or shared-memory parallel frameworks.

show abstract

“…Seven graph-based approaches are examined: k-clique communities [13], WGCNA [14], NNN [15], CAST [16], CLICK [17], maximal clique [18][19][20], and paraclique [21]. These methods use a graph approach, with genes as nodes and edges between genes defined based on a similarity measure.…”

Section: Introductionmentioning

confidence: 99%

A Systematic Comparison of Genome Scale Clustering Algorithms

Jay

Eblen

Zhang

et al. 2011

Bioinformatics Research and Applications

View full text Add to dashboard Cite

Background: A wealth of clustering algorithms has been applied to gene co-expression experiments. These algorithms cover a broad range of approaches, from conventional techniques such as k-means and hierarchical clustering, to graphical approaches such as k-clique communities, weighted gene co-expression networks (WGCNA) and paraclique. Comparison of these methods to evaluate their relative effectiveness provides guidance to algorithm selection, development and implementation. Most prior work on comparative clustering evaluation has focused on parametric methods. Graph theoretical methods are recent additions to the tool set for the global analysis and decomposition of microarray co-expression matrices that have not generally been included in earlier methodological comparisons. In the present study, a variety of parametric and graph theoretical clustering algorithms are compared using well-characterized transcriptomic data at a genome scale from Saccharomyces cerevisiae. Methods: For each clustering method under study, a variety of parameters were tested. Jaccard similarity was used to measure each cluster's agreement with every GO and KEGG annotation set, and the highest Jaccard score was assigned to the cluster. Clusters were grouped into small, medium, and large bins, and the Jaccard score of the top five scoring clusters in each bin were averaged and reported as the best average top 5 (BAT5) score for the particular method. Results: Clusters produced by each method were evaluated based upon the positive match to known pathways. This produces a readily interpretable ranking of the relative effectiveness of clustering on the genes. Methods were also tested to determine whether they were able to identify clusters consistent with those identified by other clustering methods.

show abstract

Genome-Scale Computational Approaches to Memory-Intensive Applications in Systems Biology

Cited by 59 publications

References 38 publications

Mining maximal cliques from a large graph using MapReduce: Tackling highly uneven subproblem sizes

Mining maximal cliques from a large graph using MapReduce: Tackling highly uneven subproblem sizes

Efficient Maximal Clique Enumeration Over Graph Data

A Systematic Comparison of Genome Scale Clustering Algorithms

Contact Info

Product

Resources

About