A Dichotomy for Regular Expression Membership Testing

Abstract

We study regular expression membership testing: Given a regular expression of size m and a string of size n, decide whether the string is in the language described by the regular expression. Its classic O(nm) algorithm is one of the big success stories of the 70s, which allowed pattern matching to develop into the standard tool that it is today.Many special cases of pattern matching have been studied that can be solved faster than in quadratic time. However, a systematic study of tractable cases was made possible only recently, with the first conditional lower bounds reported by Backurs and Indyk [FOCS'16]. Restricted to any "type" of homogeneous regular expressions of depth 2 or 3, they either presented a nearlinear time algorithm or a quadratic conditional lower bound, with one exception known as the Word Break problem.In this paper we complete their work as follows:• We present two almost-linear time algorithms that generalize all known almost-linear time algorithms for special cases of regular expression membership testing. * Theorem 3. For any δ > 0, if 4-Clique has no O(n 3+δ ) algorithm, then Word Break has no O(n 1+δ/3 ) algorithm for n = m.We remark that this situation of having matching conditional lower bounds only for combinatorial algorithms is not uncommon, see, e.g., Sliding Window Hamming Distance [6].New Almost-Linear Time Algorithms. We establish two more types for which the membership problem is in almost-linear time. Theorem 4. We design a deterministicÕ(n) + O(m) algorithm for | + •+-membership and an expected time n 1+o(1) + O(m) algorithm for | + •|-membership. These algorithms also work for t-membership for any subsequence t of | + •+ or | + •|, respectively.This generalizes all previously known almost-linear time algorithms for any t-membership problem, as all such types t are proper subsequences of |+•+ or |+•|. Moreover, no further generalization of our algorithms is possible, as shown below.

Fine-Grained Complexity of Analyzing Compressed Data: Quantifying Improvements over Decompress-and-Solve

“…From a technical perspective, our paper is most related to the conditional lower bounds for sequence similarity measures on strings and curves that have been shown in recent years, specifically, the SETH-based lower bounds for edit distance [10], longest common subsequence [2,17], Fréchet distance [14], and others [4,11,16,62].…”

“…The following conjectures assert that these bounds are close to optimal, and have been used e.g. in [1,16]. k-SUM In the k-SUM problem, we are given integers R, t ≥ 0 and a set Z ⊆ {0, 1, .…”

More consequences of falsifying SETH and the orthogonal vectors conjecture

“…If the conjecture is true, then dozens of important problems from all across computer science exhibit running time lower bounds that match existing upper bounds up to subpolynomial factors. These include pattern matching and other problems in bioinformatics [7, 10, 40, 1], graph algorithms [47,6,32], computational geometry [16], formal languages [11,18], time-series analysis [2,19], and even economics [42] (see [58] for a more comprehensive list).Gao et al[32] also named the low-dimension OV conjecture, which asserts that OV does not have subquadratic algorithms whenever D = ω(log N ) holds. The low-dimension implies the moderate-dimension variant of the OV conjecture, and both are implied by SETH [54].…”

Multivariate Fine-Grained Complexity of Longest Common Subsequence

“…Some input restrictions yield a discrete or even finite set of special cases. For example, Backurs and Indyk [17] and later Bringmann et al [27] studied special cases of regular expression pattern matching by restricting the input to certain "types" of regular expressions. The set of types is discrete and infinite, however, there are only finitely many tractable types, and finitely many minimal hardness results.…”