Improving Accuracy and Robustness of Self-Tuning Histograms by Subspace Clustering

Abstract

In large databases, the amount and the complexity of the data calls for data summarization techniques. Such summaries are used to assist fast approximate query answering or query optimization. Histograms are a prominent class of model-free data summaries and are widely used in database systems. So-called self-tuning histograms look at query-execution results to refine themselves. An assumption with such histograms, which has not been questioned so far, is that they can learn the dataset from scratch, that is-starting with an empty bucket configuration. We show that this is not the case. Self-tuning methods are very sensitive to the initial configuration. Three major problems stem from this. Traditional self-tuning is unable to learn projections of multi-dimensional data, is sensitive to the order of queries, and reaches only local optima with high estimation errors. We show how to improve a self-tuning method significantly by starting with a carefully chosen initial configuration. We propose initialization by dense subspace clusters in projections of the data, which improves both accuracy and robustness of self-tuning. Our experiments on different datasets show that the error rate is typically halved compared to the uninitialized version.

Glue: Adaptively Merging Single Table Cardinality to Estimate Join Query Size

“…Traditional CardEst methods, such as histogram [27] and sampling [13,16,18], are widely applied in DBMS and generally based on simplified assumptions and expert-designed heuristics. Many variants of histograms [1,3,6,8,9,15,23,25,29,30,33,34] are proposed later to enhance their performance. Sampling-based variants include query-driven kernel-based methods [13,16], index based methods [18] and random walk based methods [19,41].…”

Robust Multi-View Subspace Clustering Via Weighted Multi-Kernel Learning and Co-Regularization

“…Inspired by [34], to fully mine the unique complementary information provided by different views, the co-regularization is introduced into the problem (6). This centroid-based approach enforces representations across different views towards a common centroid.…”

“…After the development in recent years, researchers put forward multi-seed space clustering algorithms. According to the different representation methods of subspace, the existing subspace clustering algorithms can be divided into four main types: statistical methods [2], iterative methods [3] [4], algebraic methods [5] [6] [7] and spectral-type methods [8] [9]. Statistical methods, such as mixtures of probabilistic principal component analyzers (MPPCA) [2], need to know in advance the number of subspace and dimensions, when the data and the noise distribution in space is not Gaussian distribution, the solution is not optimal.…”

Framework for novel subspace clustering using search optimization methodology

“…Kim et al [12] proposed a representation for subspace as an elastic-net, a new kind of the scheme that would imply the use of singular values of elastic-net regularization.Tang et al [13] used the search method of k Nearest Neighbors (k-NN) algorithms, being important regarding the implementation of machine learning and computer vision applications. Khachatryan et al [14] showed the significant improvement in the self-tuning technique by initializing the configuration. Further to enhance the robustness and accuracy factor in self-tuning the clusters of dense subspaces were proposed in data projections.…”