Efficient random walk sampling in distributed networks

Sarma, Anish Das; Molla, Anisur Rahaman; Pandurangan, Gopal

doi:10.1016/j.jpdc.2015.01.002

Cited by 12 publications

(4 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Performing random walk generation efficiently in networks is a fundamental task. Atish et al [4], [5] proposed several fast distributed random walk algorithms, which minimize the number of rounds and messages required in distributed settings. Zhou et al [37] introduced a distributed second-order random walk algorithm on a Pregel-like graph computation framework [20], [31] to speed up the node2vec model.…”

Section: Fast Random Walk In Networkmentioning

confidence: 99%

UniNet: Scalable Network Representation Learning with Metropolis-Hastings Sampling

Yao

Shao

Cui

et al. 2020

Preprint

View full text Add to dashboard Cite

Network representation learning (NRL) technique has been successfully adopted in various data mining and machine learning applications. Random walk based NRL is one popular paradigm, which uses a set of random walks to capture the network structural information, and then employs word2vec models to learn the low-dimensional representations. However, until now there is lack of a framework, which unifies existing random walk based NRL models and supports to efficiently learn from large networks. The main obstacle comes from the diverse random walk models and the inefficient sampling method for the random walk generation. In this paper, we first introduce a new and efficient edge sampler based on Metropolis-Hastings sampling technique, and theoretically show the convergence property of the edge sampler to arbitrary discrete probability distributions. Then we propose a random walk model abstraction, in which users can easily define different transition probability by specifying dynamic edge weights and random walk states. The abstraction is efficiently supported by our edge sampler, since our sampler can draw samples from unnormalized probability distribution in constant time complexity. Finally, with the new edge sampler and random walk model abstraction, we carefully implement a scalable NRL framework called UniNet. We conduct comprehensive experiments with five random walk based NRL models over eleven real-world datasets, and the results clearly demonstrate the efficiency of UniNet over billion-edge networks.

show abstract

Section: Fast Random Walk In Networkmentioning

confidence: 99%

UniNet: Scalable Network Representation Learning with Metropolis-Hastings Sampling

Yao

Shao

Cui

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Also, the network topologies of some of the advance level resource searching methods such as flooding [13] and random walk [14] often lead to load balancing issues among the peers. Possible ways to overcome such issues are to improve the efficiencies of a notable search algorithm called random-walk sampling [15], and further by guiding the search methods through information collections [16], [17].…”

Section: Related Workmentioning

confidence: 99%

CCLBR: Congestion Control-Based Load Balanced Routing in Unstructured P2P Systems

Shen

Chang

Liu

et al. 2018

IEEE Systems Journal

View full text Add to dashboard Cite

Given the growing popularity of the peer-to-peer (P2P) network systems in the recent years, efficient query routing under highly dynamic environments is still lacking in several P2P network systems. In response to this challenge, this paper proposes a new churn-resilient system to find alternative routing paths for the purpose of balancing the query loads under higher network churns and heavy workloads, ultimately to improve the search efficiency. Two novel methods are devised to balance the network query loads among both inter-and intragroup level peers. First, a resource grouping and a rewiring method is proposed to spontaneously organize and cluster the peers having same resources together. This strategy facilitates the peers to evolve the network into a cluster-like topology and balances the query loads among the intergroup peers. Second, a collaborative Q-learning method is proposed to balance the query loads among the intragroup peers in order to intelligently avoid queries being forwarded to the congested peers in the network. Experiments conducted under dynamic network scenarios demonstrate that our proposed method achieves better search performances with a more balanced network load than the existing methods, and further exhibits higher robustness and adaptability under higher network churns and heavy network loads.Index Terms-Collaborative Q-learning, congestion control, load balancing, query routing, unstructured peer-to-peer (P2P) systems.

show abstract

“…The random walks on complex networks was of typical interest in various kinds of scientific fields, such as statistics physics, combinatorial mathematics, computer science, chemistry, social and economic science as well as biological science [10][11][12][13][14]. Random walks were not merely an effective instrument to solve various problems, and have found diverse applications in real-world networks, for example, routing [15], searching [16], sampling [17] and data collection [18,19], community detection [20,21], network synchronization [22,23], random algorithm [24,25], and so on [26][27][28].…”

Section: Introductionmentioning

confidence: 99%

Mean First-Passage Time on Scale-Free Networks Based on Rectangle Operation

Wang

et al. 2021

Front. Phys.

View full text Add to dashboard Cite

The mean first-passage time of random walks on a network has been extensively applied in the theory and practice of statistical physics, and its application effects depend on the behavior of first-passage time. Here, we firstly define a graphic operation, namely, rectangle operation, for generating a scale-free network. In this paper, we study the topological structures of our network obtained from the rectangle operation, including degree distribution, clustering coefficient, and diameter. And then, we also consider the characteristic quantities related to the network, including Kirchhoff index and mean first-passage time, where these characteristic quantities can not only be used to evaluate the properties of our network, but also have remarkable applications in science and engineering.

show abstract

Efficient random walk sampling in distributed networks

Cited by 12 publications

References 50 publications

UniNet: Scalable Network Representation Learning with Metropolis-Hastings Sampling

UniNet: Scalable Network Representation Learning with Metropolis-Hastings Sampling

CCLBR: Congestion Control-Based Load Balanced Routing in Unstructured P2P Systems

Mean First-Passage Time on Scale-Free Networks Based on Rectangle Operation

Contact Info

Product

Resources

About