Towards materialized view selection for distributed databases

Abstract

Materialized views (MV) can significantly improve the query performance of relational databases. In this paper, we consider MVs to optimize complex scenarios where many heterogeneous nodes with different resource constraints (e.g., CPU, IO and network bandwidth) query and update numerous tables on different nodes. Such problems are typical for large enterprises, e.g., global retailers storing thousands of relations on hundreds of nodes at different subsidiaries.Choosing which views to materialize in a distributed, complex scenario is NP-hard. Furthermore, the solution space is huge, and the large number of input factors results in non-monotonic cost models. This prohibits the straightforward use of brute-force algorithms, greedy approaches or proposals from organic computing. For the same reason, all solutions for choosing MVs we are aware of do not consider either distributed settings or update costs.In this paper we describe an algorithmic framework which restricts the sets of considered MVs so that a genetic algorithm can be applied. In order to let the genetic algorithm converge quickly, we generate initial populations based on knowledge on database tuning, and devise a selection function which restricts the solution space by taking the similarity of MV configurations into account. We evaluate our approach both with artificial settings and a real-world RFID scenario from retail. For a small setting consisting of 24 tables distributed over 9 nodes, an exhaustive search needs 10 hours processing time. Our approach derives a comparable set of MVs within 30 seconds. Our approach scales well: Within 15 minutes it chooses a set of MVs for a realworld scenario consisting of 1,000 relations, 400 hosts, and a workload of 3,000 queries and updates.

CB-Pattern Trees : Identifying Distributed Nodes Where Materialization is Available

“…Distributing cubes vertical fragments discussed in [7]. The approach followed the multidimensional fragmentation of the fact table and the smaller sub fact tables are distributed across nodes.…”

Optimal Materialized View Management in Distributed Environment Using Random Walk Approach

“…When the remote basic data source changes, the materialized views in data warehouse are also updated in order to maintain the consistency, this causes the need for handling the problem of materialized view maintenance [6].…”

Power efficiency through tuple ranking in wireless sensor network monitoring

“…The second tier (sensor-side), consists of a number of sensor nodes that are positioned in predefined areas of interest. The sensor devices are loaded with the KSpot client software running on the TinyOS [12] operating system. The KSpot client currently extends the TinyDB base implementation by enabling the execution of Top-k queries in the form of aggregates.…”

“…Thus, we only project the attributes related to Q prior to storing the result in the in-memory buffer V i (line 3). The next step of the algorithm merges the tuples that arrive from the children of s i into V i (line [4][5][6][7][8][9][10][11][12][13]. This yields an in-network view similar to Figure 1 …”

“…Supplementary approaches to cope with the energy challenge during query processing have in the recent years appeared at numerous venues. These approaches range from efficient join processing algorithms in sensor networks [26,14,52,33], to the underlying data management layer [12,1,21,65] and network optimization [42,15,6,2], among others. Yet, these approaches focus either on a different system model or a different problem formulation, than the work we present in this paper.…”