Pivot-based metric indexing

Abstract

The general notion of a metric space encompasses a diverse range of data types and accompanying similarity measures. Hence, metric search plays an important role in a wide range of settings, including multimedia retrieval, data mining, and data integration. With the aim of accelerating metric search, a collection of pivot-based indexing techniques for metric data has been proposed, which reduces the number of potentially expensive similarity comparisons by exploiting the triangle inequality for pruning and validation. However, no comprehensive empirical study of those techniques exists. Existing studies each offers only a narrower coverage, and they use different pivot selection strategies that affect performance substantially and thus render cross-study comparisons difficult or impossible. We offer a survey of existing pivot-based indexing techniques, and report a comprehensive empirical comparison of their construction costs, update efficiency, storage sizes, and similarity search performance. As part of the study, we provide modifications for two existing indexing techniques to make them more competitive. The findings and insights obtained from the study reveal different strengths and weaknesses of different indexing techniques, and offer guidance on selecting an appropriate indexing technique for a given setting.

An empirical assessment of quality metrics for diversified similarity searching

“…Similarity searches are usually modeled after the Metric Spaces Model [Hetland 2009;Chen et al 2017] due to its (i) distance-based semantics [Zezula et al 2010] and (ii) computational-bounded complexity [Hetland 2020]. Formally, a metric space is a pair O, δ of a data domain O and a distance function δ, which complies with the following properties for any objects o q , o i , o j ∈ O.…”

“…• 291 index-and-search algorithms [Hjaltason and Samet 2003;Chen et al 2017;Hetland 2020], but they may present a semantic drawback for searching dense datasets. For instance, suppose a composer runs a similarity search for the five most similar tunes to the "Beatles Hey Jude" in a social-network repository and retrieves versions and parodies of the same song.…”

A Novel Method to Prevent Misconfigurations of Industrial Automation and Control Systems

“…As a well-known problem in data mining, the purpose of string similarity search is to find all strings within a given edit distance from the query string in a set of strings [12], [13], [14], [15], [16], [17]. However, most related researches focus on building the index of a fixed size set of strings to improve the performance of query [14], [15], [16], [17]. Only a few works have been done on data stream, and most of them focus on time series [18], [19], and most use the sliding window model [12] which is apparently different from the landmark model used in this paper.…”

Optimization for Large-Scale Fuzzy Joins Using Fuzzy Filters in MapReduce

“…Afrati et al [29] proposed multiple algorithms to perform a fuzzy join with Hamming, Edit and Jaccard distance in a single MapReduce stage without filters. Other algorithms [30]- [32] use pivots to split data into disjoint partitions by recursive jobs.…”