Diverse near neighbor problem

Abstract

Motivated by the recent research on diversity-aware search, we investigate the k-diverse near neighbor reporting problem. The problem is defined as follows: given a query point q, report the maximum diversity set S of k points in the ball of radius r around q. The diversity of a set S is measured by the minimum distance between any pair of points in S (the higher, the better).We present two approximation algorithms for the case where the points live in a d-dimensional Hamming space. Our algorithms guarantee query times that are sub-linear in n and only polynomial in the diversity parameter k, as well as the dimension d. For low values of k, our algorithms achieve sub-linear query times even if the number of points within distance r from a query q is linear in n. To the best of our knowledge, these are the first known algorithms of this type that offer provable guarantees.

Composable core-sets for diversity and coverage maximization

“…As described in the introduction and Appendix A, our core-sets can be directly used to obtain constant-factor approximation algorithms (for the respective diversity measures) in the context of data stream computation, distributed data processing and diverse nearest neighbor search. Some of those implications are essentially known (in particular, the streaming and distributed algorithms for the remote-edge and the remote-star measures [22,12,23] and the nearest neighbor search algorithms for the remote-edge measure [2]). Other results and implications are, to the best of our knowledge, new.…”

“…The definitions and algorithm mentioned here are from [1,2] and are only included for completeness. Please see the original papers for the detailed theoretical [2] or experimental [1] analysis of its performance.…”

“…Please see the original papers for the detailed theoretical [2] or experimental [1] analysis of its performance. The algorithm uses the techniques of locality-sensitive hashing.…”

“…• Similarity search: recently, composable core-sets have been used to construct efficient near neighbor search algorithms that maximize the diversity of the answers, both in theory [2] and in practice [1]. This is done by observing that several similarity search algorithms (notably those based on the Locality-Sensitive Hashing technique) proceed by hashing each point into multiple buckets.…”

“…This correspondence is then used to bound the error incurred by the core-set. Note that for the remote-edge diversity measure, this analysis is analogous to the analysis in [2] (although that analysis was focused on the k-center clustering as opposed to the diversity maximization).…”