Indexing large metric spaces for similarity search queries

Abstract

One of the common queries in many database applications is finding approximate matches to a given query item from a collection of data items. For example, given an image database, one may want to retrieve all images that are similar to a given query image. Distance-based index structures are proposed for applications where the distance computations between objects of the data domain are expensive (such as high-dimensional data) and the distance function is metric. In this paper we consider using distance-based index structures for similarity queries on large metric spaces. We elaborate on the approach that uses reference points (vantage points) to partition the data space into spherical shell-like regions in a hierarchical manner. We introduce the multivantage point tree structure (mvp-tree) that uses more than one vantage point to partition the space into spherical cuts at each level. In answering similaritybased queries, the mvp-tree also utilizes the precomputed (at construction time) distances between the data points and the vantage points.We summarize the experiments comparing mvp-trees to vp-trees that have a similar partitioning strategy, but use only one vantage point at each level and do not make use of the precomputed distances. Empirical studies show that the mvp-tree outperforms the vp-tree by 20% to 80% for varying query ranges and different distance distributions. Next, we generalize the idea of using multiple vantage points and discuss the results of experiments we have made to see how varying the number of vantage points in a node affects search performance and how much is gained in performance by making use of precomputed distances. The results show that, after all, it may be best to use a large number of vantage points in an internal node in order to end up with a single directory node and keep as many of the precomputed distances as possible to provide more efficient filtering during search operations. Finally, we provide some experimental results that compare mvp-trees with M-trees, which is a dynamic distance-based index structure for metric domains.A preliminary version of this paper appeared in

The Basic Principles of Metric Indexing

“…A couple of variations of interest are ROAESA, or Reduced Overhead-AESA [85], and TLAESA, or Tree-LAESA [92,93], which reduce the CPU time of AESA and LAESA, respectively, to sublinear. Applications of pivoting in hybrid structures include MVP-tree [81,82], D-index [50-52, 52, 53, 101] and the related, DF-tree [37], PM-tree [84], and CM-tree [47]. 24.…”

“…For more information about the VP-tree, see the papers by Uhlmann [62] (who calls them simply metric trees) and Yianilos [95] (who rediscovered them, and gave them their name). The VP partitioning scheme has been extended in, for example, the Optimistic VP-tree [83] and the MVP-tree [81,82]. It is extended with a so-called exclusion zone in the Excluded Middle Vantage Point Forest [61] (which is in many ways very similar to the more recent D-index, discussed later).…”

“…The MVP-tree is discussed in detail by Bozkaya andÖzsoyoglu [81,82]. In addition to the basic partitioning strategy, each leaf also contains a LAESA structure, filtering with the pivots found on the path from the root to that leaf (thereby reusing distances that have already been computed).…”

Continually Answering Constraint k-NN Queries in Unstructured P2P Systems

“…In the first category, many indexing structures have been proposed to speed up the retrieval efficiency, such as R-tree [25] , K-d tree [26] , TV-tree [27] , SR-tree [28] , and X-tree [29] . If the distance function dist on D is a metric distance function, indexing structures on metric space can be applied, such as GH-tree [30] , GNAT [31] , SA-tree [32] , M-tree [33] , and MVP-tree [34] . In the second category, many efficient search algorithms have been proposed to reduce the number of disk page access during a k-NN search [1,2,4] .…”

Similarity search without tears: the OMNI-family of all-purpose access methods

“…Another method [20] is the generalized hyper-plane tree (gh-tree), which partitions the data set into two by picking two points as representatives and assigning the remaining to the closest representative. Bozkaya and Ozsoyoglu [7] [6] proposed an extension of the vp-tree called multi-vantage-point tree (mvp-tree) which chooses in a clever way m vantage points for a node which has a fanout of m 2 . The Geometric Near Access Tree (GNAT) of Brin [8] can be viewed as a refinement of the second technique presented in [9].…”