Nanocubes for Real-Time Exploration of Spatiotemporal Datasets

Abstract

Consider real-time exploration of large multidimensional spatiotemporal datasets with billions of entries, each defined by a location, a time, and other attributes. Are certain attributes correlated spatially or temporally? Are there trends or outliers in the data? Answering these questions requires aggregation over arbitrary regions of the domain and attributes of the data. Many relational databases implement the well-known data cube aggregation operation, which in a sense precomputes every possible aggregate query over the database. Data cubes are sometimes assumed to take a prohibitively large amount of space, and to consequently require disk storage. In contrast, we show how to construct a data cube that fits in a modern laptop's main memory, even for billions of entries; we call this data structure a nanocube. We present algorithms to compute and query a nanocube, and show how it can be used to generate well-known visual encodings such as heatmaps, histograms, and parallel coordinate plots. When compared to exact visualizations created by scanning an entire dataset, nanocube plots have bounded screen error across a variety of scales, thanks to a hierarchical structure in space and time. We demonstrate the effectiveness of our technique on a variety of real-world datasets, and present memory, timing, and network bandwidth measurements. We find that the timings for the queries in our examples are dominated by network and user-interaction latencies.

GPU rasterization for real-time spatial aggregation over arbitrary polygons

“…Finally, the aggregates are computed over the materialized join results and incur additional query processing costs. Data cube-based structures (e.g., [33]) can be used to maintain aggregate values. However, creating such structures requires costly pre-processing while the memory overhead can be prohibitively high.…”

“…Not surprisingly, the problem of providing efficient support for visualization tools and interactive queries over large data has attracted substantial attention recently, predominantly for relational data [1,6,27,30,31,33,35,56,66]. While methods have also been proposed for speeding up selection queries over spatio-temporal data [17,70], these do not support interactive rates for aggregate queries, that slice and summarize the data in different ways, as required by visual analytics systems [4,20,44,51,58,67].…”

AVIST: A GPU-Centric Design for Visual Exploration of Large Multidimensional Datasets

“…These methods reduce big data into smaller data. For example, Lins et al [13] propose a new data structure called nanocube to aggregate large spatiotemporal datasets for fast querying and visualization. However, due to this, data explorations are constrained by its data structure, and it cannot directly provide flexibility and low level data details (Nanocube does not allow one to query down to any individual record).…”

Generating Heat Maps of Popular Routes Online from Massive Mobile Sports Tracking Application Data in Milliseconds While Respecting Privacy

“…Previous studies of fast heat map generation with user selectable search criteria have used either one-dimensional points in space and time, e.g., [20], or spatially extended objects for density estimation [21]. Also, as far as we know, these methods do not observe privacy.…”