A sublinear algorithm for weakly approximating edit distance

Abstract

We show how to determine whether the edit distance between two given strings is small in sublinear time. Specifically, we present a test which, given two n-character strings A and B, runs in time o(n) and with high probability returns "CLOSE" if their edit distance is O(n α ), and "FAR" if their edit distance is Ω(n), where α is a fixed parameter less than 1. Our algorithm for testing the edit distance works by recursively subdividing the strings A and B into smaller substrings and looking for pairs of substrings in A, B with small edit distance. To do this, we query both strings at random places using a special technique for economizing on the samples which does not pick the samples independently and provides better query and overall complexity. As a result, our test runs in timeÕ nfor any fixed α < 1. Our algorithm thus provides a trade-off between accuracy and efficiency that is particularly useful when the input data is very large.We also show a lower bound of Ω(n α/2 ) on the query complexity of every algorithm that distinguishes pairs of strings with edit distance at most n α from those with edit distance at least n/6.

Improved lower bounds for embeddings into L1

“…It is well known that for two strings x, y ∈ V drawn independently at random, with probability Ω(1) they satisfy ed(x, y) ≥ Ω(n). (Indeed, for every x ∈ {0, 1} n , the number of strings y ∈ {0, 1} n within edit distance (say) n/10 from x is (3n) n/10 = o(2 n ); see [7,Lemma 8] and [22,Lemma 4.4].) We thus get…”

Image similarity search with compact data structures

“…If such a sketching technique exists, it gives a significant saving in storage space (since the sketches are much smaller than the original data items) as well as running time. Sketch constructions have been developed for a number of purposes, including estimating set membership [5], estimating similarity of sets [6], estimating distinct elements and vector norms [1,12], as well as estimating string edit distance [4]. It is shown in [8] how sketch constructions can be derived from rounding techniques used in approximation algorithms.…”

“…image database is that a group of researchers defined 32 sets of similar images for this image collection 4 . These sets represent different categories and have different number of similar images in each set.…”

Surprises in approximating Levenshtein distances

“…It seems possible to estimate probabilistically, with sublinear complexity, whether the L-distance of two sequences is 'small' or 'large'; see Batu et al (2003). Whether an improvement of this rather coarse result or even a replacement of the L-distance is possible, with at most linear complexity and a non-probabilistic outcome, seems open.…”