Memory-Aware Framework for Efficient Second-Order Random Walk on Large Graphs

Shao, Yingxia; Huang, Shiyue; Miao, Xupeng; Cui, Bin; Chen, Lei

doi:10.1145/3318464.3380562

Cited by 17 publications

(8 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sampling from a discrete probability distribution 𝑃 = {𝑝 0 , 𝑝 1 , ..., 𝑝 𝑛−1 } is to select an element ℎ from {0, 1, ..., 𝑛 − 1} based on 𝑃 (i.e., 𝑃 [ℎ = 𝑖] = 𝑝 𝑖 ). In this paper, we focus on five sampling techniques, including naive sampling, inverse transformation sampling [40], alias sampling [58], rejection sampling [50] and a special case of rejection sampling [65] because they are efficient and widely used [46,52,53,59,65]. Naive sampling only works on the uniform discrete distribution, while the other four can handle non-uniform and select the element ℎ in two phases: initialization, which preprocesses the distribution 𝑃, and generation, which picks an element on the basis of the initialization result.…”

Section: Sampling Methodsmentioning

confidence: 99%

“…RW algorithm optimization. Due to the importance of the RW-based applications, a variety of algorithm-specific optimizations have been proposed for different RW applications, e.g., PPR [17,35,54,60,62], Node2Vec [69] and second-order random walks [53]. In contrast, we aim to design a generic and efficient random walk framework on which users can easily implement different kinds of random walk applications.…”

Section: Related Workmentioning

confidence: 99%

“…Moreover, we build alias tables for DeepWalk in a preprocessing phase to accelerate the execution of queries. However, this method is prohibitively expensive for high order RW due the the exponential memory consumption [53,65]. For example, the space complexity of such an index for Node2Vec, which is second-order, is 𝑂 ( 𝑣 ∈𝑉 𝑑 2 𝑣 ), and it can consume more than 1000 TB space for twitter.…”

Section: Observationsmentioning

confidence: 99%

See 2 more Smart Citations

ThunderRW: An In-Memory Graph Random Walk Engine (Complete Version)

Sun,

Chen,

et al. 2021

Preprint

View full text Add to dashboard Cite

As random walk is a powerful tool in many graph processing, mining and learning applications, this paper proposes an efficient inmemory random walk engine named ThunderRW. Compared with existing parallel systems on improving the performance of a single graph operation, ThunderRW supports massive parallel random walks. The core design of ThunderRW is motivated by our profiling results: common RW algorithms have as high as 73.1% CPU pipeline slots stalled due to irregular memory access, which suffers significantly more memory stalls than the conventional graph workloads such as BFS and SSSP. To improve the memory efficiency, we first design a generic step-centric programming model named Gather-Move-Update to abstract different RW algorithms. Based on the programming model, we develop the step interleaving technique to hide memory access latency by switching the executions of different random walk queries. In our experiments, we use four representative RW algorithms including PPR, DeepWalk, Node2Vec and MetaPath to demonstrate the efficiency and programming flexibility of ThunderRW. Experimental results show that ThunderRW outperforms state-of-the-art approaches by an order of magnitude, and the step interleaving technique significantly reduces the CPU pipeline stall from 73.1% to 15.0%.

show abstract

Section: Sampling Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Observationsmentioning

confidence: 99%

See 1 more Smart Citation

ThunderRW: An In-Memory Graph Random Walk Engine (Complete Version)

Sun,

Chen,

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The use of the Alias Method for Node2Vec incurs the "memory explosion problem" [54] since the preprocessing phase for a second-order random walk on a graph with |E| edges has a support whose cardinality is O eij ∈E deg (j) , where deg(j) is the degree of the destination node of the edge e ij ∈ E. Therefore, the time and memory complexity needed for preprocessing make the Alias method intractable even on relatively small graphs. For instance, on the unfiltered Human STRING PPI graph (19.354 nodes and 5.879.727 edges) it would require 777 GB of RAM.…”

Section: Efficient Sampling For Node2vec Random Walksmentioning

confidence: 99%

GraPE: fast and scalable Graph Processing and Embedding

Cappelletti¹,

Fontana²,

Casiraghi³

et al. 2021

Preprint

View full text Add to dashboard Cite

Graph Representation Learning methods have enabled a wide range of learning problems to be addressed for data that can be represented in graph form. Nevertheless, several real world problems in economy, biology, medicine and other fields raised relevant scaling problems with existing methods and their software implementation, due to the size of real world graphs characterized by millions of nodes and billions of edges. We present GraPE, a software resource for graph processing and random walk based embedding, that can scale with large and high-degree graphs and significantly speed up-computation. GraPE comprises specialized data structures, algorithms, and a fast parallel implementation that display several orders of magnitude improvement in empirical space and time complexity compared to state of the art software resources, with a corresponding boost in the performance of machine learning methods for edge and node label prediction and for the unsupervised analysis of graphs.GraPE is designed to run on laptop and desktop computers, as well as on high performance computing clusters.Several software libraries have been developed to efficiently process and analyze graphs, including iGraph [10], GraphLab [11], GraphX [12] and SNAP [13]. In addition, several software resources implement graph embedding algorithms based on random walk sampling to map nodes and edges into a vector space [14, 15, 16].These resources, as well as other recent software libraries that implement graph neural networks [17, 18], share the problem of scalability with large graphs. Real-world networks often contain millions of of nodes 1

show abstract

“…The empirical results demonstrate that UniNet can be 10X-900X faster than the existing open-sourced versions. Compared to the state-of-the-art sampling techniques (e.g., KnightKing [35], Memory-aware sampler [32]), our M-H based edge sampler achieves 9.6%-73.2% efficiency improvement in most of the settings and can scale to much larger networks.…”

Section: Introductionmentioning

confidence: 99%

UniNet: Scalable Network Representation Learning with Metropolis-Hastings Sampling

Yao

Shao

Cui

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Network representation learning (NRL) technique has been successfully adopted in various data mining and machine learning applications. Random walk based NRL is one popular paradigm, which uses a set of random walks to capture the network structural information, and then employs word2vec models to learn the low-dimensional representations. However, until now there is lack of a framework, which unifies existing random walk based NRL models and supports to efficiently learn from large networks. The main obstacle comes from the diverse random walk models and the inefficient sampling method for the random walk generation. In this paper, we first introduce a new and efficient edge sampler based on Metropolis-Hastings sampling technique, and theoretically show the convergence property of the edge sampler to arbitrary discrete probability distributions. Then we propose a random walk model abstraction, in which users can easily define different transition probability by specifying dynamic edge weights and random walk states. The abstraction is efficiently supported by our edge sampler, since our sampler can draw samples from unnormalized probability distribution in constant time complexity. Finally, with the new edge sampler and random walk model abstraction, we carefully implement a scalable NRL framework called UniNet. We conduct comprehensive experiments with five random walk based NRL models over eleven real-world datasets, and the results clearly demonstrate the efficiency of UniNet over billion-edge networks.

show abstract

Memory-Aware Framework for Efficient Second-Order Random Walk on Large Graphs

Cited by 17 publications

References 33 publications

ThunderRW: An In-Memory Graph Random Walk Engine (Complete Version)

ThunderRW: An In-Memory Graph Random Walk Engine (Complete Version)

GraPE: fast and scalable Graph Processing and Embedding

UniNet: Scalable Network Representation Learning with Metropolis-Hastings Sampling

Contact Info

Product

Resources

About