A dataspace-based framework for OLAP analyses in a high-variety multistore

Abstract

The success of NoSQL DBMSs has pushed the adoption of polyglot storage systems that take advantage of the best characteristics of different technologies and data models. While operational applications take great benefit from this choice, analytical applications suffer the absence of schema consistency, not only between different DBMSs but within a single NoSQL system as well. In this context, the discipline of data science is steering analysts away from traditional data warehousing and toward a more flexible and lightweight approach to data analysis. The idea is to perform OLAP analyses in a pay-as-you-go manner across heterogeneous schemas and data models, where the integration is progressively carried out by the user as the available data is explored. In this paper, we propose an approach to support data analysis within a high-variety multistore, with heterogeneous schemas and overlapping records. Our approach supports relational, document, wide-column, and key-value data models by automatically handling both data model and schema heterogeneity through a dataspace layer on top of the underlying DBMSs. The expressiveness we enable corresponds to GPSJ queries, which are the most common class of queries in OLAP applications. We rely on nested relational algebra to define a cross-database execution plan. The system has been prototyped on Apache Spark.

Intelligent Storage Data Classification System Based on the BP Neural Network

“…For this study, N � 5. (0, 0), (0, 1), (1, 0), (1,1) are used to represent four kinds of stored grain pests: flat grain thief, rice elephant, pseudograin thief, and grain beetle. us, the BP network structure is as follows: there are 5 neurons in the input layer, 8 neurons in the hidden layer, and 2 neurons in the output layer.…”

“…Since 1998, China's National Grain Reserve Depot has been built for many years since the founding of new China, with the largest investment, the largest construction scale, the most complete construction contents, and the most extensive use of advanced storage technology. It has realized the great leap forward development of China's grain storage, transportation, management, and technology, further improved the layout of National Grain Reserve Depots [1], and greatly improved the infrastructure conditions of grain circulation. Most granaries can take science and technology as the guide, comprehensively promote the use of new technologies, new warehouse types, and new equipment, greatly change the backward situation of lowlevel grain storage in the past [2], lay a good foundation for improving the scienti c and technological content of grain storage facilities, and initially establish a more reasonable grain reserve system.…”

Multidimensional Data Analysis of Ambient Air Quality Based on Apache Kylin

“…The fact table refers to the table that stores the measure values and also stores the foreign keys of the dimension table, and all the data used for analysis is ultimately from the fact table. The dimension table is used to store the content, hierarchy, and other descriptive information of the dimension [2] . The so-called multidimensional data model is the interconnection between dimension tables and fact tables.…”

Unified Views for Querying Heterogeneous Multi-model Polystores

“…Combining SQL and NoSQL systems in one infrastructure, called polystore, brings the notions of multi-store, heterogeneity and data distribution. With vertical distribution where one entity class is found in one datastore of the polystore, unary operators are executed on one system and the binary operator join is executed outside DBMS with an external function [1] [5] [12]. HydRa [10], a framework, mentions entity fragmentation but do not explain how to considerate it for querying purposes.…”

“…Our rewriting system take into account data transfer and transformation and favors the use of DBMS operators as well as its performance. Declarative Forresi et al [12] Spark…”