Dynamic Graph Partitioning Scheme for Supporting Load Balancing in Distributed Graph Environments

Abstract

As dynamic graph data have been actively used, incremental graph partition schemes have been studied to efficiently store and manage large graphs. In this paper, we propose a vertex-cut based novel incremental graph partitioning scheme that supports load balancing in a distributed environment. The proposed scheme chooses the load of each node that considers its storage utilization and throughput as the partitioning criterion. The proposed scheme defines hot data that means a particular vertex frequently searched among graphs requested by queries. We manage and utilize hot data for graph partitioning. Finally, we perform vertex-cut based dynamic graph partitioning by using a vertex replication index, the load each node, and hot data to distribute the load evenly in a distributed environment. In order to verify the superiority of the proposed partitioning scheme, we compare it with the existing partitioning schemes through a variety of performance evaluations.

Graph Computing Systems and Partitioning Techniques: A Survey

“…In the recent big graph era, graphs are inherently dynamic. The graphs' topology is dynamically changed because some vertices and edges may be removed or added from the graph over time [150], [151]. As these graphs' topology evolves, the partitioning quality of partitioners would be constantly degraded due to unbalanced load distribution in each partition and communication overhead.…”

Figures of Graph Partitioning by Counting, Sequence and Layer Matrices

“…Cutting a graph into smaller pieces is one of the fundamental algorithmic operations; partitioning large graphs is often an important subproblem for complexity reduction or parallelization [14]. With (or without [15]) nonnegative weights on vertices, in [16,17] the balanced connected k-partition problem was addressed, which is known to be NP-hard. In the same context the minimum gap graph partitioning problem was formulated, as addressed in [18].…”

“…The classifiers discussed above seem perfectly fit for this task (like Table 10 contains a summary for Figure 1 as the exemplified case). 4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27, 28} U2, D2: {1, 2, 3,4,6,7,9,10,11,12,13,14,15,16,18,19,20,21,22,24,25,26,27 Regarding the pairing (U2, D2) appearing for C 28 − T d (Figure 27), this pairing does not appear for C 28 − D 2 (Figure 25 vs. Figure 26) suggesting that its ...…”

Distributed Subgraph Query Processing Using Filtering Scores on Spark

“…In the IoT, for example, data transmission and control flows among connected devices ("things") are modeled as graphs that are analyzed to identify anomalies or to group things used by the interactions. Since large amounts of graphs have been generated in respect of social media, the IoT, and so on, systems have been developed to partition and store graphs to perform distributed processing [9][10][11][12].…”