Parallel Shortest Path Graph Computations of United States Road Network Data on Apache Spark

Arfat, Yasir; Mehmood, Rashid; Albeshri, Aiiad

doi:10.1007/978-3-319-94180-6_30

Cited by 11 publications

(6 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By partitioning the original graph into several subgraphs, this method calculates the shortest path in the original graph after calculating (in parallel) the shortest path in each partition. A method to calculate the shortest path efficiently using SPARK, a parallel platform based on distributed memory, was proposed in [14]. Fan et al [15] proposed various parallelizing sequential graph computations, including the graph shortest path, which can be migrated to an existing graph system such as a MapReduce-based implementation.…”

Section: Related Workmentioning

confidence: 99%

Optimization of Shortest-Path Search on RDBMS-Based Graphs

Seo

Ahn

2019

IJGI

View full text Add to dashboard Cite

Calculation of the shortest path between two nodes in a graph is a popular operation used in graph queries in applications such as map information systems, social networking services, and biotechnology. Recent shortest-path search techniques based on graphs stored in relational databases are able to calculate the shortest path efficiently, even in large data using frontier-expand-merge operations. However, previous approaches used a sequential bidirectional search method that causes a bottleneck, thus degrading performance. The repeated use of an aggregate SQL function also degrades performance. This paper proposes a parallel bi-directional search method using multithreading. In addition, an efficient optimization method is proposed that uses B-tree indexing instead of an aggregate SQL function. Various experiments using synthetic and real data reveal that the proposed optimization technique performs more efficiently than conventional methods. As the size of data in practical applications continues to grow, these optimizations will enable the shortest path in a graph to be found quickly and accurately.

show abstract

Section: Related Workmentioning

confidence: 99%

Optimization of Shortest-Path Search on RDBMS-Based Graphs

Seo

Ahn

2019

IJGI

View full text Add to dashboard Cite

show abstract

“…The penetration of these technologies to all spheres of everyday life has given rise to the smart infrastructure developments; smart transportation infrastructure is at the forefront of these developments [1,32,33,34,35,36]. The use of GPS devices and mobile signals to collect vehicle location and congestion data [37]; the use of big data [38,39,40] and high performance computing (HPC) [38,40,41,42] technologies; mobile, cloud and fog computing [37,43,44,45,46]; image processing and artificial intelligence (AI) for traffic analysis [47]; urban logistics prototyping [48]; vehicular ad hoc networks [44,49,50,51,52]; autonomous driving [47]; autonomic transportation systems [53,54,55]; and the use of social media for traffic event detection [56,57,58] are a few examples. There is a need for innovative uses of the cutting-edge technologies in transportation.…”

Section: Introductionmentioning

confidence: 99%

Smarter Traffic Prediction Using Big Data, In-Memory Computing, Deep Learning and GPUs

Aqib

Mehmood

Alzahrani

et al. 2019

Sensors

Self Cite

View full text Add to dashboard Cite

Road transportation is the backbone of modern economies, albeit it annually costs 1.25 million deaths and trillions of dollars to the global economy, and damages public health and the environment. Deep learning is among the leading-edge methods used for transportation-related predictions, however, the existing works are in their infancy, and fall short in multiple respects, including the use of datasets with limited sizes and scopes, and insufficient depth of the deep learning studies. This paper provides a novel and comprehensive approach toward large-scale, faster, and real-time traffic prediction by bringing four complementary cutting-edge technologies together: big data, deep learning, in-memory computing, and Graphics Processing Units (GPUs). We trained deep networks using over 11 years of data provided by the California Department of Transportation (Caltrans), the largest dataset that has been used in deep learning studies. Several combinations of the input attributes of the data along with various network configurations of the deep learning models were investigated for training and prediction purposes. The use of the pre-trained model for real-time prediction was explored. The paper contributes novel deep learning models, algorithms, implementation, analytics methodology, and software tool for smart cities, big data, high performance computing, and their convergence.

show abstract

“…societies, and smart infrastructure developments [11,12]; smart transportation infrastructure is at the forefront of these developments [13][14][15]. The use of GPS devices and mobile signals to collect vehicle location and congestion data [16]; the use of big data [17][18][19][20] and high-performance computing (HPC) [17,19,21,22] technologies; mobile, cloud and fog computing [16,[23][24][25][26]; image processing, deep learning, and artificial intelligence (AI) for road traffic analysis and prediction [27][28][29][30]; urban logistics prototyping [31]; vehicular ad hoc networks [24,[32][33][34][35]; autonomous driving [27]; autonomic transportation systems [36][37][38]; and the use of social media for traffic event detection [39][40][41]; are but a few examples. There is a need for innovative uses of the cutting-edge technologies in transportation.…”

Section: Introductionmentioning

confidence: 99%

Rapid Transit Systems: Smarter Urban Planning Using Big Data, In-Memory Computing, Deep Learning, and GPUs

et al. 2019

Self Cite

View full text Add to dashboard Cite

Rapid transit systems or metros are a popular choice for high-capacity public transport in urban areas due to several advantages including safety, dependability, speed, cost, and lower risk of accidents. Existing studies on metros have not considered appropriate holistic urban transport models and integrated use of cutting-edge technologies. This paper proposes a comprehensive approach toward large-scale and faster prediction of metro system characteristics by employing the integration of four leading-edge technologies: big data, deep learning, in-memory computing, and Graphics Processing Units (GPUs). Using London Metro as a case study, and the Rolling Origin and Destination Survey (RODS) (real) dataset, we predict the number of passengers for six time intervals (a) using various access transport modes to reach the train stations (buses, walking, etc.); (b) using various egress modes to travel from the metro station to their next points of interest (PoIs); (c) traveling between different origin-destination (OD) pairs of stations; and (d) against the distance between the OD stations. The prediction allows better spatiotemporal planning of the whole urban transport system, including the metro subsystem, and its various access and egress modes. The paper contributes novel deep learning models, algorithms, implementation, analytics methodology, and software tool for analysis of metro systems.

show abstract

Parallel Shortest Path Graph Computations of United States Road Network Data on Apache Spark

Cited by 11 publications

References 32 publications

Optimization of Shortest-Path Search on RDBMS-Based Graphs

Optimization of Shortest-Path Search on RDBMS-Based Graphs

Smarter Traffic Prediction Using Big Data, In-Memory Computing, Deep Learning and GPUs

Rapid Transit Systems: Smarter Urban Planning Using Big Data, In-Memory Computing, Deep Learning, and GPUs

Contact Info

Product

Resources

About