Polypheny-DB: Towards a Distributed and Self-Adaptive Polystore

Vogt, Marco; Stiemer, Alexander; Schuldt, Heiko

doi:10.1109/bigdata.2018.8622353

Cited by 14 publications

(11 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Currently, we are working on a new low-level storage engine for Cottontail DB, which will enable more fine grained control over I/O and buffering of data pages while at the same time taking advantage of certain access patterns inherent to certain types of queries and data. We also plan to add support for SQL by integrating Cottontail DB into the polystore Polypheny DB [20]. Long-term ideas involve support for distribution to several nodes and leveraging SIMD instructions for nearest neighbour search operations through Java's project Panama 9 and JVM support for vector intrinsics.…”

Section: Discussionmentioning

confidence: 99%

Cottontail DB: An Open Source Database System for Multimedia Retrieval and Analysis

Gasser

Rossetto

Heller

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

Self Cite

View full text Add to dashboard Cite

Multimedia retrieval and analysis are two important areas in "Big data" research. They have in common that they work with feature vectors as proxies for the media objects themselves. Together with metadata such as textual descriptions or numbers, these vectors describe a media object in its entirety, and must therefore be considered jointly for both storage and retrieval. In this paper we introduce Cottontail DB, an open source database management system that integrates support for scalar and vector attributes in a unified data and query model that allows for both Boolean retrieval and nearest neighbour search. We demonstrate that Cottontail DB scales well to large collection sizes and vector dimensions and provide insights into how it proved to be a valuable tool in various use cases ranging from the analysis of MRI data to realizing retrieval solutions in the cultural heritage domain.

show abstract

Section: Discussionmentioning

confidence: 99%

Cottontail DB: An Open Source Database System for Multimedia Retrieval and Analysis

Gasser

Rossetto

Heller

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

Self Cite

View full text Add to dashboard Cite

show abstract

“…The most relevant work in the category “Fragmentation of Multimedia Databases” is [ 1 ] since it has been cited in several papers ([ 26 , 32 , 40 , 41 , 55 , 67 , 73 ]); in contrast, the most influential method in the category “Dynamic Fragmentation” is [ 70 ] because it has motivated a number of approaches ([ 28 , 29 , 30 , 31 , 32 , 35 , 52 , 56 , 60 , 66 , 69 , 70 , 75 , 77 , 79 , 91 ]); in the category “Fragmentation for NOSQL DBMS”, the article that has attracted more attention is [ 63 ]; finally, [ 25 ] is the most cited work in the category “Other Types of Fragmentation” ([ 23 , 26 , 27 , 32 ]).…”

Section: Discussionmentioning

confidence: 99%

“…Vogt, Stiemer & Schuldt [ 77 ] presented Polypheny-DB’s vision of a distributed polystore system that seamlessly combines replication and partitioning with local polystore and can dynamically adapt all parts of the system when workload changes. The basic components for both parts of the system were presented and the open challenges towards the implementation of the Polypheny-DB vision were shown.…”

Section: Description Of the Set Of Work By Categorymentioning

confidence: 99%

Application of Dynamic Fragmentation Methods in Multimedia Databases: A Review

Castro-Medina

Rodríguez-Mazahua

López-Chau

et al. 2020

Entropy

View full text Add to dashboard Cite

Fragmentation is a design technique widely used in multimedia databases, because it produces substantial benefits in reducing response times, causing lower execution costs in each operation performed. Multimedia databases include data whose main characteristic is their large size, therefore, database administrators face a challenge of great importance, since they must contemplate the different qualities of non-trivial data. These databases over time undergo changes in their access patterns. Different fragmentation techniques presented in related studies show adequate workflows, however, some do not contemplate changes in access patterns. This paper aims to provide an in-depth review of the literature related to dynamic fragmentation of multimedia databases, to identify the main challenges, technologies employed, types of fragmentation used, and characteristics of the cost model. This review provides valuable information for database administrators by showing essential characteristics to perform proper fragmentation and to improve the performance of fragmentation schemes. The reduction of costs in fragmentation methods is one of the most desired main properties. To fulfill this objective, the works include cost models, covering different qualities. In this analysis, a set of characteristics used in the cost models of each work is presented to facilitate the creation of a new cost model including the most used qualities. In addition, different data sets or reference points used in the testing stage of each work analyzed are presented.

show abstract

“…In [20], we have introduced Polypheny-DB, a novel distributed polystore database. Polypheny-DB considers distribution at two levels: At global level, data is fragmented and replicated (the latter to increase availability) in order to allocate it to different sites in a global network.…”

Section: Introductionmentioning

confidence: 99%

“…At local level, each site runs an independent polystore that can decide unilaterally, based on a local cost model, which data stores to provide, how to distribute data across these data stores, and how to process queries. Assume, as an example for an organization running such a distributed polystore, an international auction house with databases and compute centers distributed around the globe (see [20] for more details). The auction house has to jointly deal with several workloads such as Online Transaction Processing (OLTP) (for the actual auctions), Online Analytical Processing (OLAP) (for analyzes of past auctions), graph queries (for recommendations to their customers), and finally also multimedia similarity search queries (to find items and thus auctions based on the visual appearance of the former).…”

Section: Introductionmentioning

confidence: 99%

PolyMigrate: Dynamic Schema Evolution and Data Migration in a Distributed Polystore

Stiemer

Vogt

Schuldt

et al. 2021

Heterogeneous Data Management, Polystores, and Analytics for Healthcare

Self Cite

View full text Add to dashboard Cite

In the last years, polystore databases have been proposed to cope with the challenges stemming from increasingly dynamic and heterogeneous workloads. A polystore database provides a logical schema to the application, but materializes data in different data stores, different data models, and different physical schemas. When the access pattern to data changes, the polystore can decide to migrate data from one store to the other or from one data model to another. This necessitates a schema evolution in one or several data stores and the subsequent migration of data. Similarly, when applications change, the global schema might have to be changed as well, with similar consequences on local data stores in terms of schema evolution and data migration. However, the aspect of schema evolution in a polystore database has so far largely been neglected. In this paper, we present the challenges imposed by schema evolution and data migration in Polypheny-DB, a distributed polystore database. With our work-in-progress approach called PolyMigrate, we show how schema evolution and data migration affect the different layers of a distributed polystore and we identify different approaches to effectively and efficiently propagate these changes to the underlying stores.

show abstract

Polypheny-DB: Towards a Distributed and Self-Adaptive Polystore

Cited by 14 publications

References 42 publications

Cottontail DB: An Open Source Database System for Multimedia Retrieval and Analysis

Cottontail DB: An Open Source Database System for Multimedia Retrieval and Analysis

Application of Dynamic Fragmentation Methods in Multimedia Databases: A Review

PolyMigrate: Dynamic Schema Evolution and Data Migration in a Distributed Polystore

Contact Info

Product

Resources

About