Dynamic diversification of continuous data

Δρόσου, Μαρίνα; Pitoura, Evaggelia

doi:10.1145/2247596.2247623

Cited by 29 publications

(34 citation statements)

References 16 publications

(32 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Caches are used to both speed up repeated accesses to static data and to avoid recomputation by storing the results of a computation. For result diversification problem, caching has been mainly used for addressing the continuous diversity problem where diversified subsets are computed for each sliding window over data streams [33,65]. Instead of re-evaluating all the k diverse results, the proposed scheme in [33] initializes the diverse subset of the new data window using the diverse results from the previous window.…”

Section: Scalable Diversification For Data Explorationmentioning

confidence: 99%

“…For result diversification problem, caching has been mainly used for addressing the continuous diversity problem where diversified subsets are computed for each sliding window over data streams [33,65]. Instead of re-evaluating all the k diverse results, the proposed scheme in [33] initializes the diverse subset of the new data window using the diverse results from the previous window. Whereas, in [65] an interchange algorithm is proposed to update the diverse subset as new data objects arrive.…”

Section: Scalable Diversification For Data Explorationmentioning

confidence: 99%

“…In [33], a cover tree based approach is used to index the data tuples with respect to their distances among themselves. Once a cover tree is constructed over a result set, it can be exploited to retrieve k diverse tuples in O(k) time.…”

Section: Current Approaches To Efficient Diverse Set Selectionmentioning

confidence: 99%

“…Once a cover tree is constructed over a result set, it can be exploited to retrieve k diverse tuples in O(k) time. Since, in [33] cover tree is used for dynamic diversification problem it is assumed that the user query remains same while underlying data is changing. Hence, a cover tree is initially generated using query result set and is then dynamically updated as the data changes.…”

Section: Current Approaches To Efficient Diverse Set Selectionmentioning

confidence: 99%

“…The existing work on continuous diversification problem considers only a single query (e.g., [33,65]). However, in many applications a stream processing system should be able to accommodate a workload of numerous user queries [80,81], and thus successfully compute the k diverse results at different intervals of time for each of those queries.…”

Section: Diversification Of Multiple Queries Over Data Streamsmentioning

confidence: 99%

See 4 more Smart Citations

Scalable diversification for data exploration platforms

Khan¹

View full text Add to dashboard Cite

Today, data exploration platforms are widely used to assist users in locating interesting objects within large volumes of scientific and business data. In those platforms, users try to make sense of the underlying data space by iteratively posing numerous queries over large databases.Hence, data exploration platforms rely on methods for the extraction of representative data to provide users with a concise and meaningful representation of query results. That is, extracting a few tuples from a query result to provide quick insights in the potentially huge answer space.In the past few years, importance of diversification while extracting representative subsets of data has been greatly emphasized. It has been shown that diverse subsets provide more effective representation of the underlying data by minimizing redundancy and increasing coverage.Meanwhile, search results diversification adds additional cost to an already computationally expensive exploration process.In this PhD thesis, we have focused on the design, implementation and evaluation of scalable diversification algorithms and schemes for the data exploration platforms. Particularly, this research stipulates that extracting diverse representative results during data exploration requires addressing several challenges including: 1) scaling to big volumes of high dimensional data, 2) large number of users, and 3) enabling real time continuous exploration. To address those challenges we focus on two broad aspects: 1) Diversification of high dimensional large data sets and 2) Diversification of multiple user queries.The existing work conducts diversification in two steps: first compute all relevant query results, and then diversify the query results to select a small diverse subset. Similarly, all the dimensional attributes of all the data points in a query result are considered for diversification. Such a generic approach would be a performance bottleneck in high-dimensional large databases. To efficiently compute diverse subsets of query results exhibiting both high dimensionality and high-cardinality, we have proposed the Progressive Diversification Scheme. Our proposed scheme, utilizes the partial distance computations to reduce the amount of CPU and I/O incurred during query diversification. Moreover, to avoid the overhead of computing all relevant results first, we propose embedding diversification in query evaluation step by utilizing column-based data storage systems. In addition to computational cost of diversification methods, we have also considered the complexity of diversity objective function in high dimensional databases. Often, computing diverse solutions along all the dimensions is not a realistic approach. In fact, users may have some pre-specified preferences over some dimensions of the data, while expecting good coverage over the other dimensions. Motivated by that need, we propose a novel scheme, which aims to generate representative data that balance the tradeoff iii between regret minimization and diversity maximization. Our scheme...

show abstract

Section: Scalable Diversification For Data Explorationmentioning

confidence: 99%