Terra Populus’ architecture for integrated big geospatial services

Abstract

Big geospatial data is an emerging sub-area of geographic information science, big data, and cyberinfrastructure. Big geospatial data poses two unique challenges to these and other cognate disciplines. First, raster and vector data structures and analyses have developed on largely separate paths for the last twenty years and this creates an impediment to researchers utilizing big data platforms that do not promote the integration for these classes. Second, big spatial data repositories have yet to be integrated with big data computation platforms in ways that allow researchers to spatio-temporally analyze big geospatial datasets. IPUMS-Terra, a National Science Foundation cyberInfrastructure project, begins to address these challenges. IPUMS-Terra is a spatial data infrastructure project that provides a unified framework for accessing, analyzing, and transforming big heterogeneous spatio-temporal data, and is part of the IPUMS (Integrated Public Use Microdata Series) data infrastructure. It supports big geospatial data analysis and provides integrated big geospatial services to its users. As IPUMS-Terra’s data volume grows, we seek to integrate geospatial platforms that will scale geospatial analyses and address current bottlenecks within our system. However, our work shows that there are still unresolved challenges for big geospatial analysis. The most pertinent is that there is a lack of a unified framework for conducting scalable integrated vector and raster data analysis. We conducted a comparative analysis between PostgreSQL with PostGIS and SciDB and concluded that SciDB is the superior platform for scalable raster zonal analyses.

BiGeo: A Foundational PaaS Framework for Efficient Storage, Visualization, Management, Analysis, Service, and Migration of Geospatial Big Data—A Case Study of Sichuan Province, China

“…The geospatial big data distributed cluster solution is built using the open-source projects Greenplum and PostGIS [47]. Greenplum's architecture ( Figure 2) uses massively parallel processing (MPP).…”

IPUMS-Terra: integrated big heterogeneous spatiotemporal data analysis system

“…PostGIS allows for the storage and analysis of both vector and raster data types. Haynes et al (2017) discusses some of the obstacles encountered with high-performance computing using both spatial data types.…”

“…Raster summarizations were initially degrading the performance of our system as they are computationally expensive. Haynes (2017) discusses why query performance times of raster analyses with PostgreSQL vary greatly and we have implemented a data caching system to reduce additional calculations. The caching key is generated from the following variables: geographic level, temporal time point or range, and the raster variable and raster operation (e.g., minimum value, maximum value).…”

Distributed zonal statistics of big raster and vector data

“…This paper studies the zonal statistics problem which combines raster data, e.g., temperature, with vector data, e.g., city boundaries, to compute aggregate values for each polygon, e.g., average temperature in each city. This problem has several applications including the study by ecologists on the effect of vegetation and temperature on human settlement [3,4], analyzing terabytes of socio-economic and environmental data [5,6], and studying of land use and land cover classification [7]. It can also be used for areal interpolation [8] and to assess the risk of wildfires [9].…”

“…Traditional methods to process the zonal statistics problem focused on either vectorizing the raster dataset [16] or rasterizing the vector data [5]. The first approach converts each pixel to a point and then runs a spatial join with polygons using a point-in-polygon predicate [16].…”