Hashedcubes: Simple, Low Memory, Real-Time Visual Exploration of Big Data

Pahins, Cícero A. L.; Stephens, Sean A.; Scheidegger, Carlos; Comba, João Luiz Dihl

doi:10.1109/tvcg.2016.2598624

Cited by 77 publications

(50 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…To support interactive response times for analytical queries in visualization systems, compact data structures such as Nanocubes [33] and Hashedcubes [45] have been designed to store and query the CUBE operator for spatio-temporal data sets.…”

Section: Related Workmentioning

confidence: 99%

GPU rasterization for real-time spatial aggregation over arbitrary polygons

et al. 2017

View full text Add to dashboard Cite

Visual exploration of spatial data relies heavily on spatial aggregation queries that slice and summarize the data over different regions. These queries comprise computationally-intensive point-inpolygon tests that associate data points to polygonal regions, challenging the responsiveness of visualization tools. This challenge is compounded by the sheer amounts of data, requiring a large number of such tests to be performed. Traditional pre-aggregation approaches are unsuitable in this setting since they fix the query constraints and support only rectangular regions. On the other hand, query constraints are defined interactively in visual analytics systems, and polygons can be of arbitrary shapes. In this paper, we convert a spatial aggregation query into a set of drawing operations on a canvas and leverage the rendering pipeline of the graphics hardware (GPU) to enable interactive response times. Our technique trades-off accuracy for response time by adjusting the canvas resolution, and can even provide accurate results when combined with a polygon index. We evaluate our technique on two large real-world data sets, exhibiting superior performance compared to index-based approaches.

show abstract

Section: Related Workmentioning

confidence: 99%

GPU rasterization for real-time spatial aggregation over arbitrary polygons

et al. 2017

View full text Add to dashboard Cite

show abstract

“…1(a)) and visual specification ( Fig. 1(b)(c)), which are widely used [42,44,49]. However, our current implementation does not fully utilize the benefits of RSATree.…”

Section: Limitationsmentioning

confidence: 99%

“…To reduce unnecessary storage, thereby avoiding huge storage cots, preprocessing should be systematically designed on the basis of usage scenarios [35,42,44,49,63]. Thus, obtaining accurate results is not constantly required when conducting exploratory analysis.…”

Section: Introductionmentioning

confidence: 99%

RSATree: Distribution-Aware Data Representation of Large-Scale Tabular Datasets for Flexible Visual Query

Mei

Chen

Wei

et al. 2020

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

Fig. 1. RSATree facilitates fast answering of aggregate queries in large-scale tabular datasets while allowing flexible binning strategies. (a) A case built on a social network check-in dataset with 4.5 million records. A brushing and linking operation is performed by brushing workdays (Monday -Friday) and 13 hours of each day (9am -9pm) for filtering. (b) Binned scatterplot created from an airline on-time performance dataset with 180 million records. The bin width can be freely modified and instant previews are shown. (c) Capability of RSATree to generate a histogram with varied bin widths by using the same dataset as (b). As shown on the left side of (c), the distribution of "LateAircraftDelay" is relatively unbalanced. Application of log-scale binning produces a recognizable histogram (right side). Conventional approaches cannot simultaneously achieve low response time and flexible binning strategy in the three cases.Abstract-Analysts commonly investigate the data distributions derived from statistical aggregations of data that are represented by charts, such as histograms and binned scatterplots, to visualize and analyze a large-scale dataset. Aggregate queries are implicitly executed through such a process. Datasets are constantly extremely large; thus, the response time should be accelerated by calculating predefined data cubes. However, the queries are limited to the predefined binning schema of preprocessed data cubes. Such limitation hinders analysts flexible adjustment of visual specifications to investigate the implicit patterns in the data effectively. Particularly, RSATree enables arbitrary queries and flexible binning strategies by leveraging three schemes, namely, an R-tree-based space partitioning scheme to catch the data distribution, a locality-sensitive hashing technique to achieve locality-preserving random access to data items, and a summed area table scheme to support interactive query of aggregated values with a linear computational complexity. This study presents and implements a web-based visual query system that supports visual specification, query, and exploration of large-scale tabular data with user-adjustable granularities. We demonstrate the efficiency and utility of our approach by performing various experiments on real-world datasets and analyzing time and space complexity.

show abstract

“…The data cube is designed for the hierarchical aggregation of multidimensional data structures and supports rich operations such as scrolling, drilling, slicing, dicing, and rotation. It can also provide a multidimensional view of the original data and allows the user to quickly calculate the data aggregates [2,9,[18][19][20][21][22]. However, for high-dimensional cubes, most of the cube cells are empty; this results in very high storage redundancy and excessive consumption of internal/external storage.…”

Section: Data Reduction Techniques For Big Data Visualizationmentioning

confidence: 99%

MAP-Vis: A Distributed Spatio-Temporal Big Data Visualization Framework Based on a Multi-Dimensional Aggregation Pyramid Model

et al. 2020

View full text Add to dashboard Cite

During the exploration and visualization of big spatio-temporal data, massive volume poses a number of challenges to the achievement of interactive visualization, including large memory consumption, high rendering delay, and poor visual effects. Research has shown that the development of distributed computing frameworks provides a feasible solution for big spatio-temporal data management and visualization. Accordingly, to address these challenges, this paper adopts a proprietary pre-processing visualization scheme and designs and implements a highly scalable distributed visual analysis framework, especially targeted at massive point-type datasets. Firstly, we propose a generic multi-dimensional aggregation pyramid (MAP) model based on two well-known graphics concepts, namely the Spatio-temporal Cube and 2D Tile Pyramid. The proposed MAP model can support the simultaneous hierarchical aggregation of time, space, and attributes, and also later transformation of the derived aggregates into discrete key-value pairs for scalable storage and efficient retrieval. Using the generated MAP datasets, we develop an open-source distributed visualization framework (MAP-Vis). In MAP-Vis, a high-performance Spark cluster is used as a parallel preprocessing platform, while distributed HBase is used as the massive storage for the generated MAP data. The client of MAP-Vis provides a variety of correlated visualization views, including heat map, time series, and attribute histogram. Four open datasets, with record numbers ranging from the millions to the tens of billions, are chosen for system demonstration and performance evaluation. The experimental results demonstrate that MAP-Vis can achieve millisecond-level query response and support efficient interactive visualization under different queries on the space, time, and attribute dimensions.

show abstract

Hashedcubes: Simple, Low Memory, Real-Time Visual Exploration of Big Data

Cited by 77 publications

References 34 publications

GPU rasterization for real-time spatial aggregation over arbitrary polygons

GPU rasterization for real-time spatial aggregation over arbitrary polygons

RSATree: Distribution-Aware Data Representation of Large-Scale Tabular Datasets for Flexible Visual Query

MAP-Vis: A Distributed Spatio-Temporal Big Data Visualization Framework Based on a Multi-Dimensional Aggregation Pyramid Model

Contact Info

Product

Resources

About