Servicing range queries on multidimensional datasets with partial replicas

Weng, Li; Çatalyürek, Ümit V.; Kurç, Tahsin; Agrawal, Gagan; Saltz, Joel

doi:10.1109/ccgrid.2005.1558635

Cited by 8 publications

(4 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, replicas can be created by using query history as a guide [31,52] or in a more dynamic approach where replication/indexing occurs as queries are performed [19,18].…”

Section: Replication In Storage I/o Systemsmentioning

confidence: 99%

RADAR: Runtime Asymmetric Data-Access Driven Scientific Data Replication

Jenkins

Zou

Tang

et al. 2014

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Efficient I/O on large-scale spatiotemporal scientific data requires scrutiny of both the logical layout of the data (e.g., row-major vs. column-major) and the physical layout (e.g., distribution on parallel filesystems). For increasingly complex datasets, hand optimization is a difficult matter prone to error and not scalable to the increasing heterogeneity of analysis workloads. Given these factors, we present a partial data replication system called RADAR. We capture datatype-and collective-aware I/O access patterns (indicating logical access) via MPI-IO tracing and use a combination of coarse-grained and fine-grained performance modeling to evaluate and select optimized physical data distributions for the task at hand. Unlike conventional methods, we store all replica data and metadata, along with the original untouched data, under a single file container using the object abstraction in parallel filesystems. Our system results in manyfold improvements in some commonly used subvolume decomposition access patterns. Moreover, the modeling approach can determine whether such optimizations should be undertaken in the first place.

show abstract

“…For instance, replicas can be created by using query history as a guide [31,52] or in a more dynamic approach where replication/indexing occurs as queries are performed [19,18].…”

Section: Replication In Storage I/o Systemsmentioning

confidence: 99%

RADAR: Runtime Asymmetric Data-Access Driven Scientific Data Replication

Jenkins

Zou

Tang

et al. 2014

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…al. described a partial replica selection algorithm for serving range queries on multidimensional datasets [22]. Our work differs from these efforts in two important respects.…”

Section: Related Workmentioning

confidence: 99%

A location service for partial spatial replicas implementing an R-tree in a relational database

Tian

Rhodes

2016

Journal of Parallel and Distributed Computing

View full text Add to dashboard Cite

As parallel computing has become increasingly common, the need for scalable and efficient ways of storing and locating data has become increasingly acute. For years, both grid and cloud computing have distributed data across machines and even clusters at different geographic locations (sites). However not all sites need all of the data in a particular data set, or have the (perhaps specialized) processing capabilities required. These facts challenge the conventional wisdom that we should always move the computation to the data rather than the data to the computation. Sometimes the data actually required is small. In other cases, the site with specialized processing capabilities (such as a GPU equipped cluster) cannot handle the demands placed on it unless a way is found to let that cluster select the data that is actually needed, even if it is not stored locally.

show abstract

“…A challenging issue is to be able to use different levels of storage and computing hierarchy in a coordinated way to maximize the bandwidth of data retrieval and processing. A number of strategies, such as multi-level hierarchical indexing [11], partial replication [12], caching [13], and adaptive data redistribution can be employed. To support the knowledge base, we make use of three middleware systems to support the data management and processing requirements as described above of optimization based studies for waste management application.…”

Section: Storage and Management Of Large Volumes Of Datamentioning

confidence: 99%

Towards Dynamic Data-Driven Management of the Ruby Gulch Waste Repository

Parashar

Matossian

Klíe

et al. 2006

Computational Science – ICCS 2006

View full text Add to dashboard Cite

Previous work in the Instrumented Oil-Field DDDAS project has enabled a new generation of data-driven, interactive and dynamically adaptive strategies for subsurface characterization and oil reservoir management. This work has led to the implementation of advanced multiphysics, multi-scale, and multi-block numerical models and an autonomic software stack for DDDAS applications. The stack implements a Gridbased adaptive execution engine, distributed data management services for real-time data access, exploration, and coupling, and self-managing middleware services for seamless discovery and composition of components, services, and data on the Grid. This paper investigates how these solutions can be leveraged and applied to address another DDDAS application of strategic importance -the data-driven management of Ruby Gulch Waste Repository.

show abstract

Servicing range queries on multidimensional datasets with partial replicas

Cited by 8 publications

References 22 publications

RADAR: Runtime Asymmetric Data-Access Driven Scientific Data Replication

RADAR: Runtime Asymmetric Data-Access Driven Scientific Data Replication

A location service for partial spatial replicas implementing an R-tree in a relational database

Towards Dynamic Data-Driven Management of the Ruby Gulch Waste Repository

Contact Info

Product

Resources

About