A modular extensible visualization system architecture for culled prioritized data streaming

Abstract

Massive dataset sizes can make visualization difficult or impossible. One solution to this problem is to divide a dataset into smaller pieces and then stream these pieces through memory, running algorithms on each piece. This paper presents a modular data-flow visualization system architecture for culling and prioritized data streaming. This streaming architecture improves program performance both by discarding pieces of the input dataset that are not required to complete the visualization, and by prioritizing the ones that are. The system supports a wide variety of culling and prioritization techniques, including those based on data value, spatial constraints, and occlusion tests. Prioritization ensures that pieces are processed and displayed progressively based on an estimate of their contribution to the resulting image. Using prioritized ordering, the architecture presents a progressively rendered result in a significantly shorter time than a standard visualization architecture. The design is modular, such that each module in a user-defined data-flow visualization program can cull pieces as well as contribute to the final processing order of pieces. In addition, the design is extensible, providing an interface for the addition of user-defined culling and prioritization techniques to new or existing visualization modules.

An Enhanced Visualization Process Model for Incremental Visualization

“…Parallel visualization [21], [22], [23] is used to distribute the visualization process across multiple processing units and to run it concurrently. Parallelism can be achieved (among other options) by chunking and distributing the data (data parallelism) and/or by subdividing and distributing the visualization process (pipeline parallelism) [13], [23].…”

A Survey of Visualization Pipelines

“…An interactive application may show the results of a streaming visualization pipeline as they become available. Such an application can be improved greatly by prioritizing the streamed regions to process those that provide the most information first [42]. Possible priority metrics include the following.…”

Dax Toolkit: A proposed framework for data analysis and visualization at Extreme Scale

“…These control mechanisms can also be used to stream data, in pieces, through the pipeline [3]. More recent advances allow us to prioritize the streaming, thus allowing to compensate for high latency of streaming by presenting the most relevant data first [4]. This in turn has lead to multi-resolution visualization [38,57].…”