Scalable test data generation from multidimensional models

Abstract

Multidimensional data models form the core of modern decision support software. The need for this kind of software is significant, and it continues to grow with the size and variety of datasets being collected today. Yet real multidimensional instances are often unavailable for testing and benchmarking, and existing data generators can only produce a limited class of such structures. In this paper, we present a new framework for scalable generation of test data from a rich class of multidimensional models. The framework provides a small, expressive language for specifying such models, and a novel solver for generating sample data from them. While the satisfiability problem for the language is NP-hard, we identify a polynomially solvable fragment that captures most practical modeling patterns. Given a model and, optionally, a statistical specification of the desired test dataset, the solver detects and instantiates a maximal subset of the model within this fragment, generating data that exhibits the desired statistical properties. We use our framework to generate a variety of high-quality test datasets from real industrial models, which cannot be correctly instantiated by existing data generators, or as effectively solved by general-purpose constraint solvers.

Practical Constraint Solving for Generating System Test Data

“…Another data generation approach from outside the UML community is TestBlox [79]. TestBlox aims to produce large, valid, and statistically representative data sets from multidimensional models to test the performance of big-data computing platforms.…”

Applying combinatorial test data generation to big data applications

“…We note that many approaches generate large data sets to evaluate the performance of big data computing platforms [1,3,4,13,23,26,27]. These approaches are fundamentally different from test data generation approaches whose goal is to find faults that may exist in big data programs.…”

Reversing statistics for scalable test databases generation

“…The reason is that the subtle correlations between attributes are often not captured. Another line of work [2,3,4,15,9,18] addresses this problem by considering a richer set of constraints, e.g., generating a database given a workload of queries such that each intermediate result has a certain size. They constraints are typically specified in a declarative language and the use of constraint solvers is very common in these works.…”

“…Recent work [2,9,18] has proposed generating workload-aware datasets with the help of constraint solvers. However, these do not scale well to the amounts of data typically present in a customer dataset.…”