A framework for parallel map-matching at scale using Spark

Peixoto, Douglas Alves; Nguyen, Quoc Viet Hung; Zheng, Bolong; Zhou, Xiaofang

doi:10.1007/s10619-018-7254-0

Cited by 8 publications

(6 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For simplicity, we use v and (vi → vj) to denote a vertex and a directed edge of a road network, respectively. Map-matching [14,28] aligns a raw trajectory with the road network that constrains the movement of the corresponding object, and the result is a networkconstrained accurate trajectory. This process transforms each original point location into a mapped location.…”

Section: Probabilistic Map-matchingmentioning

confidence: 99%

Compression of uncertain trajectories in road networks

Huang

Chen

et al. 2020

Proc. VLDB Endow.

View full text Add to dashboard Cite

Massive volumes of uncertain trajectory data are being generated by GPS devices. Due to the limitations of GPS data, these trajectories are generally uncertain. This state of affairs renders it is attractive to be able to compress uncertain trajectories and to be able to query the trajectories efficiently without the need for (full) decompression. Unlike existing studies that target accurate trajectories, we propose a framework that accommodates uncertain trajectories in road networks. To address the large cardinality of instances of a single uncertain trajectory, we exploit the similarity between uncertain trajectory instances and provide a referential representation. First, we propose a reference selection algorithm based on the notion of Fine-grained Jaccard Distance to efficiently select trajectory instances as references. Then we provide referential representations of the different types of information contained in trajectories to achieve high compression ratios. In particular, a new compression scheme for temporal information is presented to take into account variations in sample intervals. Finally, we propose an index and develop filtering techniques to support efficient queries over compressed uncertain trajectories. Extensive experiments with real-life datasets offer insight into the properties of the framework and suggest that it is capable of outperforming the existing state-of-the-art method in terms of both compression ratio and efficiency.

show abstract

Section: Probabilistic Map-matchingmentioning

confidence: 99%

Compression of uncertain trajectories in road networks

Huang

Chen

et al. 2020

Proc. VLDB Endow.

View full text Add to dashboard Cite

show abstract

“…Other frameworks. Besides these frameworks, there have been other efforts at developing support for map matching on top of big-data processing infrastructure [5,24,26]. As an example, CloudTP is a cloud-based system for trajectory data preprocessing and exploration that leverages Azure cloud technologies for performance [5,26].…”

Section: Existing Spatial Big-data Processing Frameworkmentioning

confidence: 99%

“…Additionally, CloudTP does not address scalability, relying instead on the cloud provider's capability to offer suitable computing resources. Other related systems include DMM [4] and the Spark-based system of Peixoto et al [24]. The former is not available as open source and both systems rely on sampling when constructing the index used for data partitioning.…”

Section: Existing Spatial Big-data Processing Frameworkmentioning

confidence: 99%

GeoMatch

Zeidan

Lagerspetz

Zhao

et al. 2020

ACM/IMS Trans. Data Sci.

View full text Add to dashboard Cite

We develop GeoMatch as a novel, scalable, and efficient big-data pipeline for large-scale map matching on Apache Spark. GeoMatch improves existing spatial big-data solutions by utilizing a novel spatial partitioning scheme inspired by Hilbert space-filling curves. Thanks to its partitioning scheme, GeoMatch can effectively balance operations across different processing units and achieve significant performance gains. GeoMatch also incorporates a dynamically adjustable error-correction technique that provides robustness against positioning errors. We demonstrate the effectiveness of GeoMatch through rigorous and extensive empirical benchmarks that consider large-scale urban spatial datasets ranging from 166,253 to 3.78B location measurements. We separately assess execution performance and accuracy of map matching and develop a benchmark framework for evaluating large-scale map matching. Results of our evaluation show up to 27.25-fold performance improvements compared to previous works while achieving better processing accuracy than current solutions. We also showcase the practical potential of GeoMatch with two urban management applications. GeoMatch and our benchmark framework are open-source.

show abstract

“…A comparative study and experiments demonstrated that our framework achieved good efficiency and scalability on map-matching processing with low memory consumption. Finally, the results of our work have been accepted for publication in the DAPD journal [119].…”

Section: Efficient Map-matching At Scalementioning

confidence: 91%

“…In addition, we employ a safe boundary threshold for trajectory segmentation and replication to reduce uncertainty. Further details can be found at Section 3.2 of this thesis, and in our published work [119].…”

Section: Contributionsmentioning

confidence: 99%

A distributed in-memory database system for large-scale spatial-temporal trajectory data

Peixoto¹

Self Cite

View full text Add to dashboard Cite

Spatial-temporal trajectory data contains rich information about moving objects and phenomena, hence have been widely used for a great number of real-world applications. However, the ubiquity and complexity of spatial-temporal trajectory data has made it challenging to efficiently store, process, and query such data. Furthermore, the increasing number of users also challenges the ability of trajectory-based services and analytics to handle the query workload and response to multiple requests in a satisfactory time. Over the last few years, a new class of systems has emerged to handle large amounts of data in an efficient manner, referred as distributed in-memory database systems. These systems were designed to overcome the difficulties to scale traditional structured and unstructured data loads that some applications have to handle. Spark has became the framework of choice for large-scale low-latency data processing using distributed in-memory computation. However, Spark-based systems still lack the ability to handle several trajectory database tasks in a memory-wise manner. Some desirable feature of trajectory database systems include, data preparation and preprocessing, large-scale data storage and retrieval, and multiuser concurrent query processing. Providing a full-fledged system architecture supporting these features is challenging, and yet an issue. Firstly, trajectories are unstructured data types, coupled with spatial and temporal attributes, and organized in a sequential manner, which is hard to fit into traditional relational and spatial database systems; furthermore, trajectory data is available in a myriad of formats, each of which contains its own data schema and attributes. Moreover, trajectory datasets are highly skew and inaccurate, due to hotspots, transmission errors, and collecting devices inaccuracy, for instance. In addition, since Spark is a distributed parallel framework, we must account for data partitioning and load-balancing. In spatial and spatial-temporal databases, balanced data partitioning structures are built in a dynamic fashion as the data is consumed, nevertheless, Spark provides a read-only data structure that does not directly support adaptive partitioning after the partitioning model is constructed. Finally, data storage and query processing on top of Spark should be memory-wise, since the datasets may be too large to comfortably fit in the cluster memory; moreover, memory space may be wasted by storing unnecessary data partitions. Optimizing load-balancing and memory usage are essential to a good Spark application. Therefore, driven by the increasing interest in scalable and efficient systems for trajectory-based analytics, we propose a distributed in-memory database system for memory-wise storage and scalable processing of spatial-temporal trajectory data, with low query latency and high throughput. We build our system on top of the Spark MapReduce framework, which provides an in-memory and fault-tolerant environment for distributed parallel processing of large-scale data....

show abstract

A framework for parallel map-matching at scale using Spark

Cited by 8 publications

References 22 publications

Compression of uncertain trajectories in road networks

Compression of uncertain trajectories in road networks

GeoMatch

A distributed in-memory database system for large-scale spatial-temporal trajectory data

Contact Info

Product

Resources

About