GISQF: An Efficient Spatial Query Processing System

Abstract

Collecting observations from all international news coverage and using TABARI software to code events, the Global Database of Event, Language, and Tone (GDELT) is the only global political georeferenced event dataset with 250+ million observations covering all countries in the world from January 1, 1979 to the present with daily updates. The purpose of this widely used dataset is to help understand and uncover spatial, temporal and perceptual trends and behaviors of the social and international system. To query such big geospatial data, traditional RDBMS can no longer be used and the need for parallel distributed solutions has become a necessity. MapReduce paradigm has proved to be a scalable platform to process and analyze Big Data in the cloud. Hadoop as an implementation of MapReduce is an open source application that has been widely used and accepted in academia and industry. However, when dealing with Spatial Data, Hadoop is not equipped well and falls short as it doesnt perform efficiently in terms of running time. SpatialHadoop is an extension of Hadoop with the support of spatial data. In this paper, we present Geographic Information System Querying Framework (GISQF) to process Massive Spatial Data. This framework has been built on top of the open source SpatialHadoop system which exploits two-layer spatial indexing techniques to speed up query processing. We show how this solution outperforms Hadoop query processing by orders of magnitude when applying queries on GDELT dataset with a size of 60 GB. We show the results for three types of queries, Longitude-Latitude Point queries, Circle-Area queries, and Aggregation queries.

Dart: A Geographic Information System on Hadoop

“…Liu et al [20] employs the MapReduce framework to develop a scalable solution to the computation of a local spatial statistic (G * i (d)). GISQF [21] is another spatial query framework on SpatialHadoop [14] to offer three types of queries, Longitude-Latitude Point queries, Circle-Area queries, and Aggregation queries. Yet, there is no operation or discussion for calculating geometric median on top of Hadoop and HBase.…”

Analyzing spatial analytics systems based on Hadoop and Spark: A user perspective

“…Another example is TerraFly GeoCloud, 57 an online application designed to aid users on their analyses by issuing spatial queries according to the context of the TerraFly spatial database 58 . In addition, the Geographic Information System Query and Analytics Framework (GISQAF) 59,60 can also make use of this guideline since it performs several analytical queries on the Global Data of Events, Language and Tone (GDELT) dataset 61 . Based on our analyses, the following SASs fulfill Guideline 1: SpatialSpark, GeoSpark, GeoMesa Spark, STARK, and Elcano.…”

“…Melhor Busão extends this application by adding a feature that predicts the next bus in a specific area. Other examples are the implementation of an extension on GISQAF, 59,60 a system whose queries return events from the GDELT 61 dataset associated with spatial objects, to execute KNN queries, and the addition of distance‐based range queries to SHAHEED, 73 a system for querying and visualizing the satellite data of the Land Process Distributed Active Archive Center. Based on our analyses, GeoSpark and GeoMesa Spark are the only SASs that completely fulfill Guideline 6.…”

GISQAF: MapReduce guided spatial query processing and analytics system

“…The framework customizes and extends SpatialHadoop to index, decode, and query the GDELT georeferenced dataset and similar datasets. GISQAF extends the work in which does not consider any approach to implement spatial Data Analytics. Therefore, this paper not only considers the Query Processing (QP), but it also considers the Data Analytics (DA).…”

“…In this paper, we introduce the Geographic Information System Query and Analytics Framework (GISQAF) that extends SpatialHadoop . GISQAF achieves two objectives: query processing (QP) and data analytics (DA).…”