Fast shortest path distance estimation in large networks

Abstract

We study the problem of preprocessing a large graph so that point-to-point shortest-path queries can be answered very fast. Computing shortest paths is a well studied problem, but exact algorithms do not scale to huge graphs encountered on the web, social networks, and other applications.In this paper we focus on approximate methods for distance estimation, in particular using landmark-based distance indexing. This approach involves selecting a subset of nodes as landmarks and computing (offline) the distances from each node in the graph to those landmarks. At runtime, when the distance between a pair of nodes is needed, we can estimate it quickly by combining the precomputed distances of the two nodes to the landmarks.We prove that selecting the optimal set of landmarks is an NP-hard problem, and thus heuristic solutions need to be employed. Given a budget of memory for the index, which translates directly into a budget of landmarks, different landmark selection strategies can yield dramatically different results in terms of accuracy. A number of simple methods that scale well to large graphs are therefore developed and experimentally compared. The simplest methods choose central nodes of the graph, while the more elaborate ones select central nodes that are also far away from one another. The efficiency of the suggested techniques is tested experimentally using five different real world graphs with millions of edges; for a given accuracy, they require as much as 250 times less space than the current approach in the literature which considers selecting landmarks at random.Finally, we study applications of our method in two problems arising naturally in large-scale networks, namely, social search and community detection.

The Spanning Tree based Approach for Solving the Shortest Path Problem in Social Graphs

“…Kleinberg et al (2004) show that landmarks can be picked randomly with good theoretical results. Potamias et al (2009) build landmarks according to basic metrics with better result than in the previous work. All the above mentioned landmark-based approaches estimates the lengths of the shortest path in |S| , where L is a set of landmarks.…”

“…For instance, the breadth-first search requires at least 1.5 GB (200 000 000 * 8 bytes) of the primary memory to store all vertices, each vertex is represented by an 8 bytes long integer, in its inner queue of a graph which contains 200 million vertices. According to the performance estimation (Potamias, Bonchi, Castillo, & Gionis, 2009), it takes roughly a minute in a standard desktop computer to calculate the shortest path using the breadth-first search between two vertices in a graph that contains four million vertices and 50 million edges. While one of the most popular social networking sites, Facebook, has circa one and a half billion active users (Statista, 2015).…”

“…However, the problem is not broadly explored for social graphs. Algorithms, like landmark-based algorithms (Kleinberg, Slivkins, & Wexler, 2004;Potamias, Bonchi, Castillo, & Gionis, 2009;Zhao, Sala, Wilson, Zheng, & Zhao, 2010), which extract a set of vertices, called landmarks, can only estimate the shortest distance between two vertices, but not find the actual path. In addition, the algorithms contain a precomputation step; thus, once the graph changes, the pre-computation should be performed again.…”

Shortest Path Estimation for Graph Data Using a Relational Technique

“…M. Potamias et al [13] provided solution for calculating shortest path in a large network and used Landmarks indexes. It gives more accurate results particularly in less time using Random, Degree, Centrality strategies.…”

Online Computation of Fastest Path in Time-Dependent Spatial Networks

“…So far no optimal strategy with respect to landmark selection and random queries has been found. Specifically, landmark selection is NP-hard [26] and ALT does not guarantee to yield the smallest search spaces with respect to fastest path computations where source and destination nodes are chosen at random. Our experiments with real-world time-dependent travel-times show that our approach consumes much less storage as compared to ALT based approaches and yields faster response times (see Section 6).…”