Scalable diversification of multiple search results

Khan, H.; Δρόσου, Μαρίνα; Sharaf, Mohamed A.

doi:10.1145/2505515.2505740

Cited by 9 publications

(5 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The need for diversity is not limited to retrieval and there has been significant research in many applications [8,20,28]. In practice, diversity is a subjective phenomenon [26].…”

Section: Evaluation Measuresmentioning

confidence: 99%

Diverse Yet Efficient Retrieval using Locality Sensitive Hashing

Rao

Jain

Jawahar

2016

Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval

View full text Add to dashboard Cite

Typical retrieval systems have three requirements: a) Accurate retrieval i.e., the method should have high precision, b) Diverse retrieval, i.e., the obtained set of points should be diverse, c) Retrieval time should be small. However, most of the existing methods address only one or two of the above mentioned requirements. In this work, we present a method based on randomized locality sensitive hashing which tries to address all of the above requirements simultaneously. While earlier hashing approaches considered approximate retrieval to be acceptable only for the sake of efficiency, we argue that one can further exploit approximate retrieval to provide impressive trade-offs between accuracy and diversity. We extend our method to the problem of multi-label prediction, where the goal is to output a diverse and accurate set of labels for a given document in real-time. Moreover, we introduce a new notion to simultaneously evaluate a method's performance for both the precision and diversity measures. Finally, we present empirical results on several different retrieval tasks and show that our method retrieves diverse and accurate images/labels while ensuring 100x-speed-up over the existing diverse retrieval approaches.

show abstract

“…The need for diversity is not limited to retrieval and there has been significant research in many applications [8,20,28]. In practice, diversity is a subjective phenomenon [26].…”

Section: Evaluation Measuresmentioning

confidence: 99%

Diverse Yet Efficient Retrieval using Locality Sensitive Hashing

Rao

Jain

Jawahar

2016

Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval

View full text Add to dashboard Cite

show abstract

“…Most work on implicit query result diversification aims at finding a set of k items based on a scoring function that considers both relevance and diversity (e.g., [36]). Khan et al [22] study the problem of the concurrent diversification problem for answering multiple diversified top-k queries over static data. Angel et al [3] study the problem of diversified keyword search in documents based on user behavior model.…”

Section: Related Workmentioning

confidence: 99%

Diversity-Aware Top-k Publish/Subscribe for Text Stream

Chen

Cong

2015

Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data

View full text Add to dashboard Cite

Massive amount of text data are being generated by a huge number of web users at an unprecedented scale. These data cover a wide range of topics. Users are interested in receiving a few up-todate representative documents (e.g., tweets) that can provide them with a wide coverage of different aspects of their query topics. To address the problem, we consider the Diversity-Aware Topk Subscription (DAS) query. Given a DAS query, we continuously maintain an up-to-date result set that contains k most recently returned documents over a text stream for the query. The DAS query takes into account text relevance, document recency, and result diversity. We propose a novel solution to efficiently processing a large number of DAS queries over a stream of documents. We demonstrate the efficiency of our approach on realworld dataset and the experimental results show that our solution is able to achieve a reduction of the processing time by 60-75% compared with two baselines. We also study the effectiveness of the DAS query.

show abstract

“…Towards that end, we propose novel schemes for diversification of multiple concurrent and sequential queries [55,56,58].…”

Section: Session Level Optimizationsmentioning

confidence: 99%

Scalable diversification for data exploration platforms

Khan¹

View full text Add to dashboard Cite

Today, data exploration platforms are widely used to assist users in locating interesting objects within large volumes of scientific and business data. In those platforms, users try to make sense of the underlying data space by iteratively posing numerous queries over large databases.Hence, data exploration platforms rely on methods for the extraction of representative data to provide users with a concise and meaningful representation of query results. That is, extracting a few tuples from a query result to provide quick insights in the potentially huge answer space.In the past few years, importance of diversification while extracting representative subsets of data has been greatly emphasized. It has been shown that diverse subsets provide more effective representation of the underlying data by minimizing redundancy and increasing coverage.Meanwhile, search results diversification adds additional cost to an already computationally expensive exploration process.In this PhD thesis, we have focused on the design, implementation and evaluation of scalable diversification algorithms and schemes for the data exploration platforms. Particularly, this research stipulates that extracting diverse representative results during data exploration requires addressing several challenges including: 1) scaling to big volumes of high dimensional data, 2) large number of users, and 3) enabling real time continuous exploration. To address those challenges we focus on two broad aspects: 1) Diversification of high dimensional large data sets and 2) Diversification of multiple user queries.The existing work conducts diversification in two steps: first compute all relevant query results, and then diversify the query results to select a small diverse subset. Similarly, all the dimensional attributes of all the data points in a query result are considered for diversification. Such a generic approach would be a performance bottleneck in high-dimensional large databases. To efficiently compute diverse subsets of query results exhibiting both high dimensionality and high-cardinality, we have proposed the Progressive Diversification Scheme. Our proposed scheme, utilizes the partial distance computations to reduce the amount of CPU and I/O incurred during query diversification. Moreover, to avoid the overhead of computing all relevant results first, we propose embedding diversification in query evaluation step by utilizing column-based data storage systems. In addition to computational cost of diversification methods, we have also considered the complexity of diversity objective function in high dimensional databases. Often, computing diverse solutions along all the dimensions is not a realistic approach. In fact, users may have some pre-specified preferences over some dimensions of the data, while expecting good coverage over the other dimensions. Motivated by that need, we propose a novel scheme, which aims to generate representative data that balance the tradeoff iii between regret minimization and diversity maximization. Our scheme...

show abstract

Scalable diversification of multiple search results

Cited by 9 publications

References 10 publications

Diverse Yet Efficient Retrieval using Locality Sensitive Hashing

Diverse Yet Efficient Retrieval using Locality Sensitive Hashing

Diversity-Aware Top-k Publish/Subscribe for Text Stream

Scalable diversification for data exploration platforms

Contact Info

Product

Resources

About